GLM-5.3 against DeepSeek V4.1 Flash on 75 questions

Each question was generated from a workbook's own cells, so its answer was known before either model saw it: a figure, and where a question had one, the cell that holds it and the building's name. 4 kinds, about a quarter each:

Every question was sent exactly as the question box sends it: the system instruction, then the whole workbook as text — every cell that holds something, as ADDRESS: VALUE, sheet by sheet, about 42,000 tokens — then the question. An answer counted as right when it stated the figure, to the cent or to the count, and where the question asked for a building, named it. Citing the right cell was scored apart.

“Choose between two tables” is the one a reader is most likely to get wrong. The workbook averages its comps twice: once under the building classes switched on, and once by class with an “All Classes” column. Asked how many comps go into the averages, the first is the answer. Both models got all of these right — after one change to the instruction, made before this run, when DeepSeek V4.1 Flash took the all-classes figure in a trial.

On the same questions, 4 were answered right by GLM-5.3 alone and 0 by DeepSeek V4.1 Flash alone. That gap, -5.3 points, is real by one test and not by another: resampling whole workbooks puts it between -9.2 and -1.4 points with 95% confidence, while McNemar's exact test on the four disagreements gives p = 0.125. With 75 questions, a difference this size sits at the edge of what can be told apart.

DeepSeek V4.1 Flash was asked each question twice more. Across its three runs it was right 94.2% of the time, and gave the same verdict all three times on 93.3% of questions: its misses are mostly the same questions, not bad luck.

Its misses were of one kind. Asked for the highest 1 BR rent on a sheet of comps, it named a building at $2,295 when the sheet held one at $2,749; asked for an average over 44 rows, it gave $2,116.67 for $1,894.25. Reading a figure out of one cell it does well; holding a whole column in view while it compares, less so. GLM-5.3, thinking first, did not miss one.

Accuracy alone says which model to trust; it does not say what each right answer costs. Three ways of putting it in one number, each counting only the answers that were right:

On every one of them DeepSeek V4.1 Flash wins, by 2.1× a minute, 14× a dollar and 29× per dollar-second. They all share a blind spot: a wrong answer costs nothing in them. It simply isn't counted.

In use, a wrong answer is not free. Someone reads it, acts on it, or has to catch it. Price that, and the comparison turns on a single figure. GLM-5.3 costs $0.0162 more a question and avoids a wrong answer 5.3% of the time, so it pays for itself whenever a wrong answer costs more than $0.30. In time, it is 2.1 s slower and is worth it whenever a wrong answer costs a reader more than 39 seconds.