What post-training changed: Particle-1 against DeepSeek V4.1 Flash and GLM-5.3-Flash in an agent harness

A round is one task and one repeat: a fresh workbook per model, seeded with the task's tabs, from which the run can read its own tabs and the open web and nothing else of the account's. Each run had 15 minutes; T10, a breadth task, had 30. Every trace was checked to confirm the run used the model it names and read nothing outside its workbook and the web.

Everything but the model is held fixed: the harness, its tools, the seed tabs, the instruction text and the time limit, and the three runs of a round started together. The unit is the run; intervals come from resampling each model's runs. With 5 repeats a task, a gap in accuracy of a few points between two models would not show; a large one would.

Grading checks each row of the delivered table against the key, fetches every cited URL to see whether the page says what the row claims, and runs one web search for any value the key does not hold before calling it made up. A row the grader cannot check counts as not correct. A second grader, on a different model, graded 35 of the runs again, picked at random before grading began: over 300 rows the two agreed on 100% (Cohen's kappa 1.00).

The harness is Claude Code: the loop that gives a model tools, executes its calls and returns the results. Only the model behind it changed between runs. Each model was served through hedwigai's model endpoint, and a run that filled its context compacted it and continued.

The environment is a hedwigai workbook. Each tab is a file the harness reads and edits as it would code; the run has a shell, web search and fetch, and the seed tabs, and no other data. Each write becomes a version that records the sources it cites, so every delivered row can be traced to the page it came from. Grading used that record directly: graders fetched each cited page and checked it against the row.

Fixing the harness and the environment is what makes the comparison about the model. The same prompt, tools and stopping rule apply to all three, and the trace of each run is the complete record of what the model did with them.

Intervals are 95%, from 10,000 resamples of each model's runs. Most tasks were easy for all three: in 6 of the 12, every run of every model got every row. The means are separated by a few runs that delivered nothing usable, which is why the intervals are wide.

T10 asks for the headquarters city and the chief executive of 15 large companies as of 12 November 2026. On 11 November NVIDIA's board accepted Jensen Huang's resignation and named its CFO, Colette Kress, interim CEO; the 8-K was filed on the 12th. The answer key was written before the change and was corrected to Kress from the 8-K; the correction is logged in the dataset.

Who caught it: Particle-1 in 5 of 5 runs, DeepSeek V4.1 Flash in 3 of 5, GLM-5.3-Flash in 0 of 5. The traces say why. GLM-5.3-Flash fetched a company's own leadership page in 5 of its runs and searched for CEO news in 5; it leaned on SEC filings, and NVIDIA's latest periodic filing was signed by Huang. DeepSeek V4.1 Flash searched the news in 5 of its runs and still missed the change in 2. A table of facts "as of" a date is only as good as the last place a run looked.

A value is made up when the page it cites says otherwise and one search finds no source for it. There are 15 across 180 runs, every one below. Several are a real figure attached to the wrong date: an exchange rate from September given for November, a fiscal year's sales from the year before, a fiscal year's end date shifted by two days.

One is odder. 5 runs, from 3 different models, gave Linear's employee tender as US$64.1M and attributed it to Dealroom, whose page says $99M. Several models inventing the same number is unlikely; a source the graders did not find is likelier. It is graded as made up, because the cited page does not say it, and we have not found where it came from.

Accuracy says where a run ended. To see the path, we borrowed the idea basins of Bespoke Labs' How models explore ideas: group the things a run tries into regions of similar approach, then follow a run from region to region.

A move is a stretch of consecutive tool calls with the same purpose, such as fetching SEC filings or editing the draft. Claude Haiku named each move from a per-task list, blind to the model; then, for triples of names, it judged which two approaches are closer, asked twice with the order swapped. A t-STE embedding of those judgments places the names on a map, and k-means divides each task's map into 3 to 7 basins. On held-out judgments the maps agree 81% to 100% of the time, against about 52% by chance; 4,988 moves in all.

Counting only research moves, the ones that read sources rather than the run's own draft or filesystem: Particle-1 visited 2.87 basins a run, DeepSeek V4.1 Flash 2.80 and GLM-5.3-Flash 2.08. GLM-5.3-Flash never left its first basin in 18% of its runs, against 5% for Particle-1.

That reads like a lack of curiosity, and mostly it is not. Most of the runs that never left their first basin are T05 and T09 runs, tasks whose answer is already in the workbook: a contract in a tab, a table to put formulas on. The rest are T04 and T10 runs whose research stayed inside one of the task's broader basins. The 16 of them that delivered got 97.9% of rows right. Over every run, exploring more did not predict a better grade: runs that never left got 92.2%, the rest 90.3%.

T10 is the exception. There facts go stale, and more research moves went with higher accuracy (Spearman 0.52 over 15 runs): the runs that went on to news and leadership pages are the runs that caught NVIDIA. In T07 the relation runs weakly the other way (Spearman -0.28 over 9 delivered runs).

Most movement is going back. 79% of all switches between research basins return to a basin the run had already visited: search, fetch, back to search, the same loop at a larger scale.

The maps are less stable than the accuracy tables. The number of basins is chosen per task by the clustering, and a different choice moves each model's count by a few tenths. The ordering is stable: Particle-1 and DeepSeek V4.1 Flash visit about the same number of basins, and GLM-5.3-Flash fewer. On this measure, post-training did not change how widely the runs searched.

Nothing in these runs is special to the benchmark. Each run worked in a hedwigai workbook: it read the seed tabs, searched and fetched the web, and wrote its table back, every write kept as a new version with its sources. The harness here was Claude Code, and the same workbooks are open to whichever harness or assistant you already use.

In Claude Code, one line connects it; then /mcp to sign in:

claude mcp add --transport http hedwigai https://mcp.hedwigai.com

Claude Desktop and claude.ai add hedwigai from the Claude directory. Grok, ChatGPT, Cursor, Codex and any other assistant that takes a remote MCP server connect to the same address, https://mcp.hedwigai.com; the agents page has the steps for each. Your assistant can then make a workbook, read and write its tabs, or start a research run on it, and every edit lands as a version you can check against its sources and undo.

Every run is on Hugging Face as hedwigai/model-bench-v2: the delivered documents, the full traces, both graders' grades, the answer keys with the T10 correction logged, and the rubric, under CC BY 4.0. The traces are complete tool-call records, for anyone studying how post-training shows up in agent behaviour.