Can forward passes alone turn a small model into a router?

An AI agent spends much of its day on small, repeated decisions. It sorts a ticket. It picks a tool. It routes a request. A large model makes these decisions well and slowly. A small model is fast and cheap. It usually gets them wrong.

We asked whether a small model can be shaped for one such job without training. There are no gradients and no optimiser. There are only forward passes and a least-squares solve. We call the result a minion.

The model is Qwen3.5-0.8B. The task comes from MASSIVE. Each request to a voice assistant belongs to one of 18 scenarios, such as alarm, music or weather. We scored 600 test requests.

The short answer is yes. With 20 examples a scenario, the minion routes 83.2% of the requests correctly. The unedited model routes 8.7%.

Every layer of the model ends with an MLP. Its neurons read the residual stream, and each writes one direction back into it. A new neuron can be added by appending one row to two input matrices and one column to the output matrix.

We add one neuron per label to the last MLP. The neuron is meant to fire only where the model is about to answer which label a request belongs to. When it fires, it writes toward the label’s first token.

Every part of the neuron comes from a least-squares solve. Shaping takes four steps.

Three numbers remain to choose: the ridge penalty, the sharpness of the gate and the strength of the output. We tried 63 combinations on a separate set of 360 requests and kept the best one. Every combination had to leave at least 99% of next-token guesses unchanged on ordinary text.

The neurons sit in the last layer, and the label is read at the last position. So each candidate’s answer follows exactly from numbers already collected. The search needs no further forward passes.

We gave the model the 18 scenario names in its prompt, then the request. The unedited model routes 8.7% of the requests correctly.

One labelled example per scenario in the prompt helps a great deal. It raises the score to 55.5%. The prompt then carries 18 extra requests and their answers.

The minion keeps the short prompt and routes 83.2%. Five examples a scenario already give 74%. Fifty give 84.7%.

Our first edit had no gate. It was a plain low-rank change to the last MLP’s output matrix, with the same keys and the same label directions.

It routed 76.5% of the test requests. It also fired at every position of every text. The model got 0 of 30 capitals right, and 0.16% of next-token guesses on ordinary text stayed the same.

We then fitted the same edit to stay at zero on 7,920 positions of ordinary text. No strength gave both jobs. A weak edit kept 89% of the text and routed 57%. A strong one routed 81% and kept almost none of it.

The gate fixes this. Its average value is 0.994 where a routing answer goes and -0.001 on ordinary text. It never rises above 0.5 on ordinary text.

With the gate, the minion gets 26 of 30 capitals right. The unedited model gets 27. Of 16,906 next-token guesses on Wikipedia text that the search never saw, 99.25% stay the same.

A gain can come from the size of an edit rather than its content. So we added 18 random neurons, each row and column scaled to the size of the fitted ones.

They route 5.5% of the requests, less than the unedited model. They also do harm. The model then gets 7 of 30 capitals right, and 80.14% of next-token guesses on ordinary text stay the same.

The fitted neurons route because of what they read and what they write.

All of the work is forward passes and matrix solves. There is no backward pass, and there is no optimiser state. The memory needed is what the model needs to run.

On one NVIDIA L4, all 1,430 forward passes took 9.2 seconds. They read the fitted examples twice, the held-aside requests and the ordinary text. The search over 63 candidate edits took 4 seconds.

The minion is the base model and a patch of 108 KB. Many minions can therefore share one copy of the base model, with each patch applied when its request arrives.

The method needs only examples and a few seconds. An agent already has both. It can collect the decisions it makes again and again, shape a minion from them, and hand the job over.

We built the shaper as a program an agent can call. It takes labels and examples, and it returns a scorecard and the patch. The scorecard always includes the unedited model and random neurons, scored on examples the fit never saw. So an agent cannot fool itself about how good its minion is.

Two limits matter. The minion’s probabilities are not calibrated. They sit near 1, even when it is wrong. An agent cannot yet use them to decide when to answer a request itself. We also tested routing only. A label is a choice the model can already express. We did not test skills the model lacks.