Blog
hedwigaihedwigaihedwigai

Writing

Frontier models by scale, 2021–2026

In November 2021 the largest language-model training run on record used about 2×10²⁴ floating-point operations (FLOP). In September 2026 GPT-6 Astra was estimated at 9.4×10²⁶, about 460 times more. This post splits five years of language models into bands of training compute and follows each band through time. Scaling laws describe one band against another at a fixed date. Here the question is what happens to each band as the dates move.

Setup

All data comes from Epoch AI’s public datasets, downloaded on 15 November 2026. The AI Models dataset gives each model’s release date and an estimate of its training compute. We kept language and multimodal models released from 15 November 2021 to 29 October 2026.

Capability is the Epoch Capabilities Index (ECI). Epoch fits one item-response model to 57 benchmarks, so models tested on different benchmarks land on one scale. The scale is anchored at 130 for Claude 3.5 Sonnet and 150 for GPT-5. 256 models have an ECI score; the first was released in February 2023. 104 of them also have a compute estimate, and those 104 carry every band result below. For 2021 and 2022 we use MMLU, a 57-subject multiple-choice exam, because the ECI does not reach back that far.

We cut training compute into four bands, each ten times the one below:

BandTraining computeModels with ECITypical members
Smallbelow 10²³ FLOP16LLaMA-13B, Phi-2, phi-3-mini
Mid10²³ to 10²⁴29Llama 2-70B, Mixtral 8x7B, Phi-4, Qwen3-30B-A3B
Large10²⁴ to 10²⁵42Llama 3.1-70B, DeepSeek-V3, Kimi K2, DeepSeek V4
Frontier10²⁵ and above17GPT-4, Claude 3.5 Sonnet, Grok 4, GPT-5, GPT-6 Astra

We measure four things. The first is the growth of the largest training runs. The second is each band’s slope: ECI points gained per year, from a least-squares line through the band’s models. The third is catch-up time: the months between the first model in a higher band and the first in a lower band to reach the same score. The fourth is a joint fit of ECI on compute and date, which prices one year of progress in compute.

The largest training run grew 460-fold, about 3× a year

Figure 1 shows the 363 language and multimodal models in the dataset trained with at least 10²³ FLOP. At the start the record was Google’s FLAN 137B, from September 2021, at 2.0×10²⁴. It was then passed by GPT-3.5 and Minerva (2.6 and 2.7×10²⁴, 2022), GPT-4 (2.1×10²⁵, March 2023), Gemini 1.0 Ultra (5.0×10²⁵, December 2023), Grok 3, GPT-4.5 and Grok 4 (3.5 to 5.0×10²⁶, 2025), and GPT-6 Astra (9.4×10²⁶, September 2026).

A line through the five largest models of each year rises 3.0× a year. The record moved in steps, not smoothly. It stood still for 14 months after Gemini 1.0 Ultra and again for 14 months after Grok 4.

10²³10²⁴10²⁵10²⁶10²⁷20222023202420252026FLOPGopherGPT-4Gemini 1.0 UltraGrok 3GPT-6 Astra
10²³–10²⁴10²⁴–10²⁵10²⁵ and above
Figure 1. Training compute of every language and multimodal model of at least 10²³ FLOP released from November 2021 to October 2026, on a log scale. Shade marks the band. Source: Epoch AI, AI Models dataset.
Models of at least 10²³ FLOP released each year, by band. 2021 counts from 15 November; 2026 runs to 29 October.
Year10²³–10²⁴10²⁴–10²⁵10²⁵–10²⁶10²⁶ and above
20212200
202220500
2023511340
2024734160
2025406963
202611881

The busiest band moved up one decade in 2025. Most of the 10²⁴–10²⁵ models that year are open-weight mixtures of experts, such as DeepSeek-V3 (3.3×10²⁴) and Qwen3-235B-A22B (4.8×10²⁴). The 2026 counts are low because most closed labs no longer publish enough for Epoch to estimate compute: only 15 of the 87 models scored in 2026 have an estimate.

In 2021 and 2022, the same compute already bought more

Before the ECI begins, MMLU shows the first sign of the pattern. Gopher scored 60.0% in December 2021. Chinchilla, released by DeepMind in March 2022, used about the same compute (5.8×10²³ FLOP) on a 70-billion-parameter model with four times as much data. It scored 67.5%. PaLM 540B used 2.5×10²⁴ FLOP, four times more than Chinchilla, and scored 69.3%. Then GPT-4, at 2.1×10²⁵, scored 86.4%.

The 10²³–10²⁴ band kept climbing on MMLU without leaving its compute range:

DateModelTraining compute (FLOP)MMLU
Dec 2021Gopher (280B)6.3×10²³60.0%
Mar 2022Chinchilla (70B)5.8×10²³67.5%
Jul 2023Llama 2-70B8.1×10²³69.9%
Nov 2023Yi-34B6.1×10²³76.3%
Dec 2024Phi-49.3×10²³84.8%

MMLU stops being useful once the best models pass 85%, because the remaining questions include mislabelled ones. From 2023 on we use the ECI.

Every band gains 12 to 20 ECI points a year

Figure 2 plots the 104 models with both an ECI score and a compute estimate. Each band forms its own rising cloud. The four fitted lines are close to parallel. The dashed line is the best score of any model to date, including closed models with no compute estimate.

60801001201401601802023202420252026
below 10²³ FLOP10²³–10²⁴10²⁴–10²⁵10²⁵ and abovebest of any model
Figure 2. Epoch Capabilities Index against release date for 104 models with a training-compute estimate, February 2023 to October 2026. Solid lines are least-squares fits within each band. The dashed line is the highest ECI of any model released so far, including the 152 without a compute estimate.
Slope of the fitted line in each band, with its standard error, and the band's best model.
BandModelsECI points per yearPeriodBest model (ECI)
below 10²³ FLOP1612.6 ± 6.32023-02 to 2024-11phi-3-mini 3.8B (110.6)
10²³–10²⁴2919.9 ± 2.92023-02 to 2025-08Qwen3-30B-A3B-Thinking (Jul 2025) (138.3)
10²⁴–10²⁵4215.2 ± 0.82023-07 to 2026-09DeepSeek V4.1 Flash (156.7)
10²⁵ and above1711.6 ± 1.22023-03 to 2026-10GPT-6 Astra (169.0)

The two well-sampled bands, 10²⁴–10²⁵ and 10²⁵ and above, climb at 15.2 and 11.6 points a year. The 10²⁵ line is pulled down by Grok 3 and GPT-4.5, two very large runs that scored below smaller models of the same months. If we fit only the models that set a new best within their band, the slopes are 20.4, 15.0, 14.2 and 13.1 points a year, from the smallest band to the largest. The smaller bands rise at least as fast as the largest one.

The small band (below 10²³ FLOP) ends in November 2024. After that, no model in the dataset that small has both a compute estimate and an ECI score, so its slope rests on 16 models and has the widest error.

A band ten times smaller reaches the same score 3 to 13 months later

Parallel lines mean a fixed delay. The table gives the first model in each band to reach each score. Read across a row to see the delay.

First model in each band to reach the score, with its release month.
ECI10²⁵ and above10²⁴–10²⁵10²³–10²⁴Delay, top two bands
120GPT-4, Mar 2023Llama 3-70B, Apr 2024Phi-4, Dec 202413 months
130Claude 3.5 Sonnet, Jun 2024DeepSeek-V3, Dec 2024Qwen3-30B-A3B, Apr 20256 months
140Claude 3.7 Sonnet, Feb 2025DeepSeek-R1 (May 2025)not yet3 months
150GPT-5, Aug 2025DeepSeek-V4-Pro, Apr 2026not yet9 months
160GPT-6 Astra, Sep 2026not yet (best 156.7)not yet2+ months so far

The delay between the top two bands fell from 13 months at ECI 120 to 3 months at ECI 140, then rose to 9 months at 150. Two bands apart, the delay is longer. GPT-4 reached 120 with 2.1×10²⁵ FLOP in March 2023. Phi-4 reached 129 with 9.3×10²³ FLOP, 23 times less, 21 months later.

The mid band (10²³–10²⁴) has no model past 138.4. That is Qwen3-30B-A3B-Thinking, from July 2025, the band’s last model with a compute estimate. Labs still release models this size, but without the figures Epoch needs, so the row ends there.

One year of progress is worth about 17 times the compute

We fit one line through all 104 models: ECI against log₁₀ of training compute and release date together. The fit explains 89% of the variance (R² = 0.888).

TermEstimate
ECI points per tenfold increase in training compute12.2
ECI points per year at fixed training compute15.2
Compute worth one year of progress17.5× (95% bootstrap interval 7.8× to 59×)

Dividing the two coefficients gives the compute a year of progress is worth: 15.2 ÷ 12.2 = 1.24 decades, or 17.5×. The largest training runs grew 3.0× a year over the same period. By this measure, what a fixed compute budget can do grew faster than the budgets themselves.

The 17.5× is not pre-training efficiency alone. Epoch’s compute estimates count pre-training and leave out most post-training. Reinforcement learning on reasoning, which arrived with OpenAI’s o1 in September 2024 and DeepSeek-R1 in January 2025, raised scores at almost unchanged estimated compute. DeepSeek-V3 and DeepSeek-R1 share the estimate of 3.3×10²⁴ FLOP; their ECI scores are 131.7 and 138.6.

A lab's smallest model trails the best model by about seven months

Since 2025, compute estimates are missing for most closed models, so the bands thin out. Labs still publish a size ordering through their product tiers. Anthropic ships Haiku, Sonnet and Opus. OpenAI ships nano, mini and full models, and since July 2026 Luna, Terra and Sol. Google ships Flash-Lite, Flash and Pro. We took the 22 small-tier and 22 middle-tier models from those three labs. For each one we counted the months since any model first reached its ECI score.

0481216202420252026
small tiermiddle tier (open circles)
Figure 3. Months between each smaller-tier model's release and the release of the first model of any lab to reach the same ECI score. Filled: small tier (Haiku, Instant, mini, nano, Flash-Lite, Luna). Open: middle tier (Sonnet, Flash, Terra). Anthropic, OpenAI and Google, June 2023 to October 2026.
YearSmall tier: modelsMedian months behindMiddle tier: modelsMedian months behind
2023350—
20245854
20258782
20266895
All227223.5

The small tier has stayed about seven months behind for three years. The delay has not shrunk. At 14 ECI points a year, seven months is about eight points. Claude Haiku 5.5, released in October 2026, scored 160.4: the level GPT-5.5 Pro first reached in April 2026, five and a half months earlier. GPT-5.4 Nano, March 2026, scored 146.4, the level of o3 eleven months before. Gemini 3.5 Flash-Lite, July 2026, is the farthest behind at 15 months.

The middle tier sits closer, at a median of 3.5 months. Claude 3.5 Sonnet was the best model of any lab when it was released in June 2024. One small-tier model was too: o1-mini, a reasoning model, in September 2024.

The top changes hands every six weeks, and the field sits closer to it

Figure 4 looks at all 256 scored models, with or without a compute estimate. Each week, the field is every model released in the year before. We rescale each model in the field so that the week’s best model scores 1 and its weakest scores 0, then smooth each curve over about two weeks. A curve starts at its model’s release and ends a year later. The 20 models that were best in the field for at least three weeks are drawn in full and labelled, and so is Claude Fable 5.2, best in the latest week. The dashed line is the median model of the field.

00.250.50.751202420252026LLaMA-65BLLaMA-33BLLaMA-13BLLaMA-7BCerebras-GPT-13BDolly 2.0-12bvicuna-13b-v1.1Falcon-7BRedPajama-INCITE-7B-BaseMPT-7BPaLM 2-LPaLM 2-MPaLM 2-SFalcon-40Bopen_llama_7bGPT-4 (Jun 2023)GPT-3.5 Turbo (Jun 2023)MPT-30Bchatglm2-6bXGen-7Binternlm-7bClaude 2Llama 2-70BLlama 2-13BLlama 2-34BLlama 2-7BStable Beluga 2Claude InstantFalcon-180BBaichuan2-13BPhi-1.5internlm-20bBaichuan 2-7BQwen-14BMistral 7B v0.1Qwen-7BDeepSeek Coder 33BDeepSeek Coder 6.7BGPT-3.5 Turbo (Nov 2023)Yi 6BQwen-1_8BMixtral 8x7BPhi-2GPT-3.5 Turbo (Jan 2024)StarCoder 2 15BStarCoder 2 7BGemma 7BGemma 2BStarCoder 2 3BMistral LargeNemotron-4 15BClaude 3 SonnetYi-9BClaude 3 HaikuCodeQwen1.5-7BMixtral 8x22BLlama 3-70BLlama 3-8Bphi-3-small 7.4Bphi-3-medium 14Bphi-3-mini 3.8BDeepSeek-V2 (MoE-236B, May 2024)Falcon 2 11BGemini 1.5 Pro (May 2024)Gemini 1.5 Flash (May 2024)Qwen2-72BDeepSeek-Coder-V2-Lite-BaseGemma 2 27BGemma 2 9BGPT-4o miniMistral NeMoLlama 3.1-405BLlama 3.1-70BLlama 3.1-8BMistral Large 2 (Jul 2024)GPT-4o (Aug 2024)o1-previewQwen2.5-Coder-32BQwen2.5-Coder-14BQwen2.5-Coder (7B)Qwen2.5-Coder-3BQwen2.5-Coder (1.5B)Qwen2.5-Coder-0.5BQwen2.5-72BGemini 1.5 Pro (Sept 2024)Gemini 1.5 Flash (Sep 2024)Llama 3.2 90BClaude 3.5 Sonnet (October 2024)Claude 3.5 HaikuGPT-4o (Nov 2024)INTELLECT-1Llama 3.3 70BGrok-2 (Dec 2024)Phi-4DeepSeek-V3DeepSeek-R1Gemini 2.0 Flash Thinking (Jan 2025)Qwen2.5-Maxo3-miniGemini 2.0 ProGemini 2.0 Flash (Feb 2025)Claude 3.7 SonnetGPT-4.5QwQ-32BGemma 3 27BDeepSeek-V3 (Mar 2025)Llama 4 ScoutLlama 4 MaverickGrok 3GPT-4.1GPT-4.1 miniGPT-4.1 nanoo4-miniGemini 2.5 Flash (Apr 2025)Qwen3-235B-A22BQwen3-8BQwen3-32BQwen3-14BQwen3-30B-A3BGemini 2.5 Pro (May 2025)Mistral Medium 3Gemini 2.5 Flash (May 2025)Claude Opus 4Claude Sonnet 4DeepSeek-R1 (May 2025)Gemini 2.5 Pro (Jun 2025)Gemini 2.5 Flash (Jun 2025)Gemini 2.5 Flash-Lite (Jun 2025)Grok-3 miniGrok 4Kimi K2 (Jul 2025)Qwen3-235B-A22B-Thinking (Jul 2025)Qwen3-235B-A22B-Instruct (Jul 2025)Qwen3-30B-A3B-Instruct (Jul 2025)Qwen3-30B-A3B-Thinking (Jul 2025)Claude Opus 4.1gpt-oss-120bgpt-oss-20bGPT-5 miniGPT-5 nanoDeepSeek-V3.1Grok 4 FastQwen3-MaxGemini 2.5 Flash (Sep 2025)Claude Sonnet 4.5DeepSeek-V3.2-ExpClaude Haiku 4.5Kimi K2 ThinkingGPT-5.1Claude Opus 4.5DeepSeek-V3.2GPT-5.2Gemini 3 FlashGLM-4.7Kimi K2.5Claude Opus 4.6GLM-5Qwen3.5 397B-A17BQwen 3.5 Plus (hosted 397B-A17B)Claude Sonnet 4.6Grok 4.20Gemini 3.1 ProQwen3.5-35B-A3BQwen3.5-9BQwen 3.5 Flash (hosted 35B-A3B)Gemini 3.1 Flash-LiteGPT-5.4GPT-5.4 MiniGPT-5.4 NanoQwen 3.6 PlusGemma 4 31B ITGemma 4 26B A4BGLM-5.1Muse SparkQwen 3.6 35B-A3BClaude Opus 4.7Grok 4.3 BetaKimi K2.6Qwen 3.6 Max (Preview)Qwen3.6 27BGPT-5.5DeepSeek-V4-ProDeepSeek-V4-FlashQwen 3.6 FlashMistral Medium 3.5Gemini 3.5 FlashQwen3.7-MaxClaude Opus 4.8MiniMax-M3Qwen3.7-PlusNemotron 3 UltraKimi K2.7 CodeGLM-5.2Claude Sonnet 5Grok 4.5GPT-5.6 SolGPT-5.6 TerraGPT-5.6 LunaMuse Spark 1.1Inkling-SmallInklingKimi K3Gemini 3.6 FlashGemini 3.5 Flash-LiteDeepSeek V4 Flash 0731Qwen 3.8 MaxMuse Spark 1.2Grok 4.6Gemini 3.7 FlashDeepSeek V4 Pro 0813GLM-5.3Qwen 3.8 27BGLM-5.3-FlashClaude Fable 5.1Qwen3.8 Max (0902)Gemini 3.8 FlashMuse Spark 1.3DeepSeek V4.1 FlashGrok 4.7Claude Opus 5.5GPT-6 SolGPT-6 LunaClaude Sonnet 5.5GPT-6.1 SolGemini 4 ArgonBeam 1.0Mistral Large 4 PreviewClaude Haiku 5.5Muse WonderMuse Spark OpenDeepSeek V4.1 Flash 1017Kimi K3.1 ProGPT-6.1 TerraGemini 4 CarbonMistral Large 4GPT-4 (Mar 2023): best in the field for 45 weeksGPT-4 Turbo (Nov 2023): best in the field for 5 weeksClaude 3 Opus: best in the field for 6 weeksGPT-4 Turbo (Apr 2024): best in the field for 4 weeksGPT-4o (May 2024): best in the field for 6 weeksClaude 3.5 Sonnet: best in the field for 12 weekso1-mini: best in the field for 14 weekso1: best in the field for 14 weeksGemini 2.5 Pro (Mar 2025): best in the field for 3 weekso3: best in the field for 8 weekso3-pro: best in the field for 8 weeksGPT-5: best in the field for 9 weeksGPT-5 Pro: best in the field for 6 weeksGemini 3 Pro: best in the field for 3 weeksGPT-5.2 Pro: best in the field for 12 weeksGPT-5.4 Pro: best in the field for 7 weeksGPT-5.5 Pro: best in the field for 7 weeksClaude Fable 5: best in the field for 6 weeksClaude Opus 5: best in the field for 6 weeksGPT-6 Astra: best in the field for 8 weeksClaude Fable 5.2: best in the field for 1 weeksGPT-4GPT-4 Turbo NovOpus 3GPT-4 Turbo AprGPT-4oSonnet 3.5o1-minio1Gemini 2.5 Proo3o3-proGPT-5GPT-5 ProGemini 3 ProGPT-5.2 ProGPT-5.4 ProGPT-5.5 ProFable 5Opus 5GPT-6 AstraFable 5.2
OpenAIAnthropicGoogleAlibabaMetaDeepSeekxAIMistralother labsmedian of the field
Figure 4. Every model with an ECI score, March 2023 to November 2026, rescaled each week between the weakest (0) and the best (1) model released in the previous twelve months, and smoothed with a two-week Gaussian. Bold: models that were best for three weeks or more, and the latest week's best. Faint: all others. Dashed: the field's median. Colour marks the lab. Hover a line for its name.

GPT-4 held the top for 45 weeks, from March 2023 to January 2024. No model has held it as long since. From 2025 on, the top changed hands 14 times, and the median hold was six and a half weeks. The longest was 12 weeks, by GPT-5.2 Pro from December 2025. OpenAI models were on top for 153 of the 193 weeks, Anthropic models for 31 and Google models for 7.

The field closed in on the top through 2025. The scaled median rose from 0.54 in 2023 to 0.76 in 2025. In ECI points, the best model led the median model by 32.5 in 2023, 23.6 in 2024, 15.6 in 2025 and 15.9 in 2026.

In 2026 the scaled median fell to 0.47 while the gap in points held at about 16. The difference is the floor. Min-max scaling depends on the weakest model in the field. Through most of 2025 that was Qwen2.5-Coder-0.5B, at 68.4. It left the field in September 2025 and INTELLECT-1, at 83.5, took its place. When INTELLECT-1 left at the end of November, the floor jumped to 126.1 (Llama 3.3 70B). Each time every curve dropped at once, but no model got worse. Read the curves for order and timing, and the point gaps for distance.

Discussion

Over five years, scale and time traded against each other at a steady rate. Each tenfold step in training compute is worth about 12 ECI points. Each year is worth about 15. The frontier band leads, and the bands below follow on nearly parallel lines. The gap between the top two bands has varied from 3 to 13 months with no clear trend. The gap between a lab’s small tier and the best model has stayed near seven months.

For someone choosing a model, the practical reading is direct. A model one tier down is, on this index, the model one tier up from about half a year earlier. A model trained with a tenth of the compute is a frontier model from three to thirteen months before.

Limits

  • Training compute is Epoch AI’s estimate, not a figure labs report. Only 104 of 256 scored models have one, and only 15 of 87 in 2026. Since 2025 the 10²⁴–10²⁵ band is mostly open-weight models from DeepSeek, Moonshot and Alibaba, so it describes those labs more than the field.
  • Compute estimates mostly leave out post-training. The joint fit therefore credits reasoning post-training to the passage of time.
  • Each ECI score has an uncertainty; the median 90% interval is 2.6 points either way. Delays measured from a single first model to cross a threshold move by months when one score moves by one point.
  • The ECI measures benchmarks. It does not measure speed, price, context length or behaviour in long agent runs.
  • Tiers were assigned from model names. OpenAI’s o1-mini, o3-mini and o4-mini count as small tier, though each was near the top of the index when released.
  • AI Models, Epoch AI — release dates, parameters and training-compute estimates.
  • Epoch Capabilities Index, Epoch AI — ECI scores and the benchmark results behind them; MMLU results from the same benchmarking hub.
  • All fits, delays and medians: our own computation over those files, downloaded 15 November 2026.

hedwigai

More head-to-head evals

Particle-1, DeepSeek V4.1 Flash and GLM-5.3-Flash in Claude Code with hedwigai workbooksParticle-1, DeepSeek V4.1 Flash and GLM-5.3-Flash, 180 runs graded blind, rows right: 91.2%, 89.4% and 90.8%, intervals overlapping.
MeasureParticle-1DeepSeek V4.1 FlashGLM-5.3-Flash
Rows right, blind91.2%89.4%90.8%
Median seconds per run375315144
Read the comparison
GLM-5.3 against DeepSeek V4.1 Flash on 75 questionsTwo models answer the same 75 questions; one is always right, the other 14× cheaper.
MeasureGLM-5.3DeepSeek V4.1 Flash
Right, of 757571
Seconds a question, median3.91.8
Read the comparison
How we made a long-running AI agent for knowledge work 4× fasterOne cited knowledge work task through 13 experiments: 937 s to 233 s, with the model held constant.
MeasureFirst runBest since
Landscape, seconds937233
Memo, seconds759602
Read the report
Ten models plan a change to a spreadsheet: which do it in under a minute, traced call by callTen models drive one agent harness through the same spreadsheet request; six write the plan, timed and traced call by call.
MeasureQwen3 235BGLM-5.2Gemini 3.8 FlashClaude Opus 5
Seconds to the plan, median27485794
Plans written, of three runs2333
Read the comparison
SubQ, jev, Perplexity's Decider and Bespoke's Nimble on biotech diligence memos: finding the sentences a column is aboutFour decision models judge every sentence of ten biotech diligence memos for five columns, scored against sentences a person marked.
MeasureSubQjevDeciderNimble
Found, mean of five columns at section length64%65%66%68%
Mechanism of action, found at section length57%51%61%54%
Read the comparison
Leads with a reason to write: hedwigai ask, Apollo and treg on one brief40 leads each, every one checked: the right person, an inbox that accepts, and a dated reason to write.
MeasureaskApollotreg
Usable leads13 of 407 of 406 of 28
Cost per usable lead$0.037$0.27$0.23
Read the comparison
Self-hosted decisions for ask: Perplexity's open Decider in place of jev on 92 company questionsThe decisions inside hedwigai ask made by Perplexity's open Decider on two NVIDIA RTX 3090s, against jev.
Measureask with jevask with Decider
Questions right, of 928989
Wrong answers12
Read the comparison
Clinical decisions: jev and ask on System One, PubMedQA, SciFact and a 370-decision setEdgeEvals' System One board, PubMedQA, SciFact, and a clinical set built to System One's rules.
Measurejevask
Our 370 clinical decisions, right94.6%93.0%
PubMedQA, right (best EdgeEvals scored: 62.2%)77.0%74.4%
Read the comparison
Executive lookup for US-listed companies: Claude Opus 5.5 and ask on 95 questionsWho runs a public company, and how to email them: a frontier model from memory against hedwigai ask.
MeasureaskClaude Opus 5.5
Questions right, of 955963
Small companies right, of 20139
Read the comparison
Rental comps: floor-plan rents and amenities on 14 listings, five assistants comparedRent per floor plan and amenities for US rentals: hedwigai realestate-us against Claude, GPT, Gemini and Perplexity with web search.
Measurerealestate-usBest of four assistants
Floor-plan rents right, of 262623
Amenities claimed the listing doesn't state12129
Read the comparison
Company screening from SEC filings: jev, Vela 2.0 and ask on 92 yes/no questionsYes-or-no questions about public companies, answered by two decision models alone and by hedwigai ask.
Measureaskjev
Right, of the questions answered98%79%
Wrong answers, of 92219
Read the comparison
Defect localization in industrial images: four frontier models and a trained localizerFour frontier models asked to put a box around what they just described.
MeasureTrained localiserBest of four frontier models
Mean overlap with the defect (IoU, 0 to 1)0.4210.109
Read the comparison