The scale of visual intelligence

We took a structured-extraction-from-images task from a 235-billion-parameter teacher down to a 2-billion-parameter student without losing accuracy. The 2B reads as well as the 8B of the same Qwen vision-language family once it can see the digits: on this task, intelligence scale saturates early. Perception does not. This post is how we found that saturation point, and why we stopped there. The 2B now serves in place of the 8B, 2.4× faster on the same GPU.

Nervous systems span eight orders of magnitude, and each one is big enough for the life it leads. A nematode finds food and avoids harm with 302 neurons. A honeybee navigates, learns flowers and dances directions with under a million.

A synapse is the closest biological cousin of a model parameter: a learned connection strength. The analogy is loose, but the lesson carries over. Evolution does not give a bee a mouse’s brain, because a bee’s tasks are already saturated at a million neurons.

We use intelligence scale to mean a model’s parameter count. For any one task, accuracy rises with intelligence scale and then goes flat. The flat part is the saturation point: past it, more parameters buy nothing for that task.

The practical question is where a given task saturates. Answer it, and you can serve the smallest model that does the job, on cheaper hardware, at lower cost. Our claim, from one vision task: the curve can look unsaturated when the real limit is somewhere else. Fix that limit, and a small model sits on the plateau.

A vision model has a second scale. We use perception scale to mean how finely the model samples its input: here, how many vision patches cover each character. The two scales saturate separately. A model can sit past its intelligence saturation point yet short of its perception one, and then its errors look like a lack of intelligence.

The task is structured extraction from photographs. Given a photo and a list of fields, the model returns each field’s exact value as JSON. The test set has photographs with digital, analog and hybrid data, with 18 values per photograph set to capture.

The photographs are hard: faded signals, shadows and oblique angles.

We did not start small. The first reader was Qwen3-VL-30B-A3B, a mixture-of-experts model with about 3B parameters active per token. The teacher was Qwen3-VL-235B-A22B, which labelled the training photographs. Scored on about 510 check-verified fields:

The student scored above its teacher. It trained on the teacher’s answers after they were checked and repaired, not on the raw answers.

From here the reference is that 8B student, fine-tuned with LoRA on about 4,400 labelled photographs. On 50 held-out photograph sets, counting all 613 values, it reads 94.6% correctly.

We distilled the 8B into Qwen3-VL-4B and Qwen3-VL-2B: same training data plus 156 photographs the 8B labelled, three epochs each. The 4B reached 93.3% and the 2B 91.5%. On its own, that looks like the familiar story: halve the parameters, lose accuracy.

The errors told a different story. Of the 2B’s 52 wrong values, 35 differed from the truth by a single digit: 8 read as 0, 5 as 3, 6 as 4. The model found the right field and read the right number of characters. It confused shapes that differ by a few pixels.

The 2B’s decoder was doing its job; the evidence never reached it. We replayed 22 distinct misread digits, each paired with a neighbouring digit the model read correctly, and measured the model’s preference for the right digit over the wrong one.

Three readings follow:

A digit in the hybrid signal of our photographs is smaller than the patch the vision encoder reads it through. Qwen3-VL cuts an image into 16×16-pixel patches, then merges each 2×2 block into one token for the language model.

So an 8 and a 0 that differ in a few pixels can land in the same patch and blur together before the decoder ever sees them. Upsampling the photo 2× gives each digit about 2×2 patches. The pixels are the same; the encoder simply samples them finely enough.

This is a sensory limit, not a cognitive one. A hawk does not need a bigger brain to spot a mouse; it needs a denser retina.

At the original resolution, accuracy climbs with size and looks unsaturated. Read the same photos at 2×, with no retraining, and every size lands on the same plateau: 96.7% for the 2B and the 8B, 97.4% for the 4B. For this task, the plateau starts at or below 2 billion parameters.

Set side by side, the two scales move accuracy by different amounts:

Quadrupling parameters at 1× gains 3.1 points. Doubling the sampling of the same 2B gains 5.2. At 2× the intelligence axis is flat: the 4B gains 0.7 points over the 2B, and the 8B reads exactly what the 2B reads, 593 of 613 values. The perception axis was the one still climbing.

Retraining the 2B on 2× photographs added nothing: 96.4% against 96.7% for the same model simply shown 2× photographs. The gain is all in what the model is given to look at.

Real pixels did no better than resampled ones. The original photographs carry 1.4× the pixels of the copies above. Read at the same image-token budget they score 96.6%; enlarged 2×, at about 7,800 tokens, 97.2%, three values more. Perception saturates too, at about this budget.

With half of its MLP neurons switched off, the 8B still reads 98.3% of digits right. The 2B, cut the same way, falls to 34.9%. The larger model is not using its size on this task.

We gave all three models the same 150 photographs and the same target text (the 8B’s answers) and recorded every layer on about 13,700 answer tokens. Then we switched parts off and measured what was lost.

The task lives in about 120 to 150 directions, whatever the width; the participation ratio is about 38 in all three. The 8B spends 4,096 dimensions on what fits in 151. Its extra size buys spare copies, not new features: half its MLP can go and the reading holds, where the 2B has no slack at all. In every model only the first layer is irreplaceable.

Two cautions. The target text is the 8B’s own, so its starting accuracy (99.8%) is flattered; compare the drops, not the starting points. And the activations are dense: 74–82% of neurons carry 90% of the activation mass in every model, so the spare capacity shows only when parts are removed, not when activity is counted.

On the same GPU, the 2B at 2× first read a photograph set in 29 s, slower than the 8B’s 24 s. A serving engine took it to 10 s.

The time was not in the model. Per photograph, the vision encoder took 1.0 s and the prefill 0.4 s; then decoding ran at 25 ms a token. Reading 2B bf16 weights once per token takes about 5 ms on that card. The other 20 ms was the generation loop launching hundreds of small GPU kernels from Python, one after another. A large model hides that overhead behind real work in each kernel; a 2B model cannot.

A serving engine (vLLM) records a whole decode step once as a CUDA graph and replays it with a single launch. It also batches the three photographs of a set as they arrive, and drops each one as its answer ends. A set then takes about as long as its longest photograph, not the sum. Batching in the old loop made things worse (38 ms a token), because it pads every answer to the longest.

A serving engine is slow to start: loading weights and capturing graphs took about 105 s, and the reader scales to zero between sets. The fix is a GPU memory snapshot of the warmed engine, taken while it sleeps, with its weights parked in CPU memory and its empty KV cache dropped. A cold start restores the snapshot and wakes it.

The answers did not move. The served 2B makes the same 20 errors as the offline run. Between runs, about 6 of 1,250 raw values differ, all on faint photographs, because results depend slightly on which requests share a batch.

The GPU time billed per photograph set fell from about 120 s for the 8B to 45–65 s for the 2B, so a set costs less than half as much. Most of what remains is the platform keeping each container up about 44 s after its last read, whatever idle window we set.

Warm, the served 2B spends about 1.75 s on each photograph before its first output token, then 12 ms per token. Nearly all of that 1.75 s is the GPU looking at the photograph.

At 2× a photograph becomes about 22,000 vision patches and 6,000 tokens. By our estimate that is around 9×10¹³ operations, which keeps an A10 busy for most of those 1.45 s.

We tried three more ways to go faster without losing accuracy. None was worth shipping:

A smaller student would not help much either. By our estimate about 70% of the work per photograph is the vision encoder, which is the same size in every model of the family. Shrinking the language model below 2B would save perhaps 3 s a set and restart the accuracy question.

In production, the first clean requests on the 2B took 27 s and 33 s end to end from a cold start, against 64–131 s with the 8B. About 10 s of that is reading; most of the rest is the GPU container starting.

Before buying a bigger brain, check the eyes. If a smaller model’s errors are near-misses on fine detail, its limit is perception, and resolution is cheaper than parameters.