Ten models plan a change to a spreadsheet: which do it in under a minute, traced call by call

The task

How long each took

Seconds are the run's own clock, for the runs that wrote a plan. "Within a minute" adds the 8 seconds before the run starts, so it counts runs a person would have seen finish in 60 seconds or less. Red dots are runs that delivered nothing.

Where the seconds went

• One long call decides the slow runs. In GLM-5.2's 96-second run, one call took 49.8 s, 56% of its model time; that is the call that writes the plan. Gemini's longest calls took 23.6 s and 23.9 s, Opus's 20.0 s and 21.4 s. Qwen3 235B's longest took 3.4 s.
• The same model can be slow in different places. GLM-5.2's 45-second run spent only 10.0 s in the model but 30.8 s in its 14 tool calls, where its slower run spent 3.9 s in 10.
• Opus thinks longest per call: a median of 10.8 s and 8.3 s a call, against 1.5 s for Qwen3 235B. It makes few calls, so it still beats the models that make many.

How they worked

• What they had in common. Every run that wrote a plan loaded the plan's method, used no web search, and wrote the same single step. Prompt tokens a call were close across models (13k to 25k): most of each call is the conversation so far, not the model's own words.
• File tools or the shell. Qwen3 235B, Gemini and GLM-5.2 worked through the harness's own read, list and write tools and used the shell little or not at all. DeepSeek did nearly everything in the shell: 18 and 16 commands a run, and 23 and 21 tool calls, the most of the models that finished in time.
• Terse or wordy. Gemini wrote 868 and 841 tokens across a whole run and Qwen3 235B 744; Opus, GLM-5.2 and Hunyuan wrote 3,000 to 8,000. Only Gemini was both terse and among the slower: it spends its time in the one long call, not in writing.
• Wandering. Hunyuan loaded 13 methods in each run, most of them for researching a company in depth rather than for planning, and in the run that delivered nothing it searched the web 6 times though it had been told not to.

How they failed

• Never wrote. GPT-6 Luna, GLM-5.3, GLM-5.3 Flash, gpt-oss-120b each left the plan exactly as it was handed over, for as long as they ran. gpt-oss-120b, served fast, stopped after nine seconds having loaded no method; GPT-6 Luna used all 180. None of the four can drive this harness, and hedwigai's workbooks no longer offer them.
• Refused by a check. One DeepSeek run wrote a complete, correct plan, then gave the written method as one of its sources. A method is an instruction, not evidence, and hedwigai refuses a delivery that cites one, so the plan was not kept. The model could do the work; its citations could not pass.
• Ran out of time. Hunyuan's slower runs reached the 180-second limit. One had written its plan by then and kept it; the other had spent the time loading methods and searching, and had written nothing.
• Left no record. One Qwen3 235B run delivered nothing, and nothing about why survived: a run's trace was held only in the server's memory until the next deploy, and a run that wrote no version left nothing behind. Since then every run keeps its own record as it goes, finished or not, so the next failure like it can be read.

What can still go wrong

• This is a small test. One request, three runs a model for six models and one for four. The same model varied by a factor of two between runs, so the order of the middle models could change with more.
• Speed depends on who serves the model. Each model was reached through hedwigai's portal and its provider at the time; another provider of the same model can be faster or slower.
• One harness. Every model drove Claude Code. A model that fails here may work in a harness written for it, and the other way round.
• An easy right answer. The plan was one step. Requests that come to many steps, in an order, will separate the models on correctness, which this one did not.