Blog
hedwigaihedwigaihedwigai

A program, measured

One YouTube link, read nine ways: from the key shots to the transcript, each priced

A transcript is one thing a video holds. It also has a look, a palette, a sequence of shots, people with names and faces, text on screen and a soundtrack. An agent researching a competitor’s launch video, a creator’s style or a product demo needs some of those and rarely all of them.

youtube describe is a program in the shell your agent gets from hedwigai. Give it a public YouTube link and a model watches the video and answers in JSON, with only the parts you ask for. We ran each of its nine parts on our own two-minute explainer, then priced the look of a fifteen-minute speech on four models, on 3 November 2026.

Each aspect asked alone of a 1:57 video, in tokens and US dollars billed
AspectWhat it givesModelTokens in / outCost
visualsThe look, the palette and each key shot3.1-flash-lite10,875 / 784$0.0097
summaryThree to five sentences3.1-flash-lite10,759 / 151$0.0073
storyThe beats, each with its time3.1-flash-lite10,767 / 346$0.008
charactersWho appears, how they look, their role3.1-flash-lite10,794 / 94$0.0071
settingWhere and when3.1-flash-lite10,753 / 88$0.0071
on_screen_textTitles, captions and slides, timed3.1-flash-lite10,770 / 1,752$0.0133
toneMood, pace and style3.1-flash-lite10,757 / 77$0.007
soundMusic and notable sound3.1-flash-lite10,762 / 72$0.007
transcriptEvery spoken line, timed, by speaker2.5-flash34,702 / 939$0.0179
All nine, one callThe video is read once2.5-flash34,997 / 2,628$0.0239

Most of every row is the video itself. Before it writes a word, the model reads the whole thing as input tokens, about 11,000 for these two minutes, so each of the eight aspects on the lighter model costs about seven tenths of a cent whether it writes 72 tokens or 1,752. Asked one by one, the nine cost $0.0844; asked together, $0.0239. Ask for everything you will need in one call.

Key visuals

The look of a video, without paying for its words

--aspects visuals returns the style in two sentences, the main colours by name, and one entry per shot: when it starts, the kind of shot, what is in frame, where things sit and the light.

$ youtube describe https://youtu.be/1rlHNULj7M4 \
    --aspects visuals \
  | jq '.description.visuals.key_shots[0]'
{
  "t": "0:00",
  "shot": "screen capture",
  "shows": "The article's title and introduction
            text, with the narrator in the bottom
            right corner.",
  "composition": "The article text is on the left,
            and the narrator is in a small circle
            in the bottom right.",
  "lighting": "Bright, even lighting."
}
  1. 0:00

    The article's title and introduction text

    Text on the left, the narrator in a small circle bottom right

  2. 0:16

    A graph titled 'Why agents plateau'

    Graph in the centre, text above and below

  3. 0:33

    Diagrams of 'Inside one session' and 'Across sessions'

    Diagrams in the centre

  4. 0:42

    A flowchart of how agents hand work to each other, and a list of roles

    Flowchart at the top, roles below

  5. 0:54

    A bar chart comparing the agent setups

    Chart in the centre

  6. 1:07

    A list of version changes, v3 to v10

    List in the centre

That is a storyboard of the explainer, written from the frames, for under a cent. On the Stanford commencement address, fifteen minutes of a man at a podium, the same call described “a medium shot of the speaker at a podium and occasional cutaways to the audience”, lit “bright and natural, consistent with an outdoor daytime event”, in red, white, black and brown, across 15 key shots from the Stanford logo to the graduates clapping.

Which model watches

Why the look and the transcript use different models

Gemini’s 2.x models read a video at about 297 tokens a second, a frame each second plus the audio. The Gemini 3 models read the same video at about 88. On a fifteen-minute speech that is 267,185 tokens against 79,795 before any answer is written, which is why the model matters more than the question on a long video.

The look of a 15:04 speech (--aspects visuals) on four models
ModelTokens in / outKey shotsCostNote
gemini-2.5-flash267,185 / 12,519169$0.156asked for at most 40 shots, gave 169
gemini-2.5-flash-lite267,185 / 3,96161$0.071gave the length as 14:50
gemini-3.1-flash-lite79,795 / 1,02715$0.054the default for anything but a transcript
gemini-3.8-flash79,795 / 3,40831$0.182the sharpest look, at 3.4 times the price

gemini-3.1-flash-lite gave a usable look for a third of what 2.5-flash charged, and 2.5-flash ignored the cap of 40 shots it was given, so the program now cuts the list at 40 itself. Its transcript was another matter: it voiced the on-screen title as the narrator’s first line and put a sentence at 0:13 that is not spoken there. So the program chooses for you when you do not name a model: gemini-2.5-flash for an answer with a transcript, gemini-3.1-flash-lite for any other. Name one with --model to override it.

Story, people, text

What happens, who is in it, what it says on screen

The story comes back as beats with their times, which reads as a chapter list. Characters are named only when the video names them, so our narrator is “a man with a beard and sunglasses”. On-screen text is the one light aspect that writes a lot: our explainer scrolls an article, and the model copied it down, timed, in 1,752 tokens.

$ youtube describe https://youtu.be/1rlHNULj7M4 \
    --aspects story \
  | jq -r '.description.story[] | "\(.t)  \(.beat)"'
0:00  The video discusses a research paper about
      how coding agents can be improved by
      working in teams with a supervisor.
0:16  The video explains that agents often plateau
      because they perform local searches and fail
      to make multiple necessary changes at once.
0:33  The video highlights that agents often forget
      information between sessions, which can be
      addressed by a supervisor.
...

All the way to the transcript

Every spoken line, timed

--transcript prints only the transcript, a line per sentence or two, and asks for nothing else, so nothing else is paid for:

$ youtube describe https://youtu.be/1rlHNULj7M4 \
    --transcript | head -5
0:00 speaker: Give a strong coding agent a hard
     GPU kernel, a thousand dollars and several
     days.
0:06 speaker: On one of DeepSeek-V3's attention
     kernels, Codex reached 28% of what the
     experts wrote.
0:14 speaker: Claude Code stopped at about half.
0:16 speaker: The agent isn't dumb.
0:18 speaker: It does local search and every edit
     is judged by the score it gets right away.

That run spelled every name the way the screen does. Not every run did. In two of the four transcripts we took of this video, the model wrote names the way they sound: “Cloud code” for Claude Code, “deep seed V3s”, “a birthly paper” for a Berkeley one, and “the head of block” for our own blog.

So the transcript is now told to spell names, products and technical terms as the screen shows them or as they are usually written, never as they sound. Run again on the same model, the same call wrote Claude Code, DeepSeek-V3, “a Berkeley paper”, Qwen and Mac, for the same $0.018. The one name it still heard was ours, which the video says but never shows: “the Headwigga blog”. A name that is only spoken is still worth checking.

Sound was the weakest aspect. A music bed runs under the narration the whole way through, and no gap in the audio falls below -30 dB, yet three of the four answers we checked said there was no music. Only the first run heard “calm, ambient background music”. Treat sound as a hint.

Long videos

A fifteen-minute video, from an agent that cannot wait

Watching takes a minute or two, and not in proportion to the length: the two-minute explainer took a little under two, the fifteen-minute speech about a minute and a half. Many MCP clients drop a tool call after about 100 seconds, so the program does not hold the call open. It starts the watching as a job, waits up to 80 seconds, and if the answer is not ready it says how to pick it up:

$ youtube describe \
    https://www.youtube.com/watch?v=UF8uR6Z6KLc \
    --aspects visuals --model gemini-2.5-flash
job: 421a887e287146f8861262ae58d9874d
still watching after 77s; nothing more is billed
for asking again:
  youtube wait 421a887e287146f8861262ae58d9874d

$ youtube wait 421a887e287146f8861262ae58d9874d \
  | jq -c '.usage'
{"cost_usd":0.1560342,"input_tokens":267185,
 "output_tokens":12519}

Add hedwigai to your MCP client from mcp.hedwigai.com, or sign in with a work email. Turn the program on with programs enable youtube, then hand your agent a link. It works on public videos; a private or age-restricted one says it cannot be watched.

More head-to-head evals

GLM-5.3 against DeepSeek V4.1 Flash on 75 questionsTwo models answer the same 75 questions; one is always right, the other 14× cheaper.
MeasureGLM-5.3DeepSeek V4.1 Flash
Right, of 757571
Seconds a question, median3.91.8
Read the comparison
How we made a long-running AI agent for knowledge work 4× fasterOne cited knowledge work task through 13 experiments: 937 s to 233 s, with the model held constant.
MeasureFirst runBest since
Landscape, seconds937233
Memo, seconds759602
Read the report
Ten models plan a change to a spreadsheet: which do it in under a minute, traced call by callTen models drive one agent harness through the same spreadsheet request; six write the plan, timed and traced call by call.
MeasureQwen3 235BGLM-5.2Gemini 3.8 FlashClaude Opus 5
Seconds to the plan, median27485794
Plans written, of three runs2333
Read the comparison
SubQ, jev, Perplexity's Decider and Bespoke's Nimble on biotech diligence memos: finding the sentences a column is aboutFour decision models judge every sentence of ten biotech diligence memos for five columns, scored against sentences a person marked.
MeasureSubQjevDeciderNimble
Found, mean of five columns at section length64%65%66%68%
Mechanism of action, found at section length57%51%61%54%
Read the comparison
Leads with a reason to write: hedwigai ask, Apollo and treg on one brief40 leads each, every one checked: the right person, an inbox that accepts, and a dated reason to write.
MeasureaskApollotreg
Usable leads13 of 407 of 406 of 28
Cost per usable lead$0.037$0.27$0.23
Read the comparison
Self-hosted decisions for ask: Perplexity's open Decider in place of jev on 92 company questionsThe decisions inside hedwigai ask made by Perplexity's open Decider on two NVIDIA RTX 3090s, against jev.
Measureask with jevask with Decider
Questions right, of 928989
Wrong answers12
Read the comparison
Clinical decisions: jev and ask on System One, PubMedQA, SciFact and a 370-decision setEdgeEvals' System One board, PubMedQA, SciFact, and a clinical set built to System One's rules.
Measurejevask
Our 370 clinical decisions, right94.6%93.0%
PubMedQA, right (best EdgeEvals scored: 62.2%)77.0%74.4%
Read the comparison
Executive lookup for US-listed companies: Claude Opus 5.5 and ask on 95 questionsWho runs a public company, and how to email them: a frontier model from memory against hedwigai ask.
MeasureaskClaude Opus 5.5
Questions right, of 955963
Small companies right, of 20139
Read the comparison
Rental comps: floor-plan rents and amenities on 14 listings, five assistants comparedRent per floor plan and amenities for US rentals: hedwigai realestate-us against Claude, GPT, Gemini and Perplexity with web search.
Measurerealestate-usBest of four assistants
Floor-plan rents right, of 262623
Amenities claimed the listing doesn't state12129
Read the comparison
Company screening from SEC filings: jev, Vela 2.0 and ask on 92 yes/no questionsYes-or-no questions about public companies, answered by two decision models alone and by hedwigai ask.
Measureaskjev
Right, of the questions answered98%79%
Wrong answers, of 92219
Read the comparison
Defect localization in industrial images: four frontier models and a trained localizerFour frontier models asked to put a box around what they just described.
MeasureTrained localiserBest of four frontier models
Mean overlap with the defect (IoU, 0 to 1)0.4210.109
Read the comparison