Which LLM, at Which Effort? Building an Apples-to-Apples Answer

Every included standard reasoning-dial configuration, sorted by quality: 30 printed, 12 projected, 0 cost-estimated, and 7 tier-4 estimated. For GPT-5.6, this means AA's standard-mode low-to-max rows, not none or pro. Claude Opus 5's dial is low/medium/high/xhigh/max, default high, and AA scored all five; its thinking-disabled mode ships only at low, medium, and high, and AA scores no row for it. Gemini 3.6 Flash's dial is minimal/low/medium/high; AA scored its top setting, high. Printed means both axes come from the current Artificial Analysis payload. Projected rows use vendor effort curves, with quality bars and roughly ±40% cost-shape uncertainty. Tier-4 means both axes come from cross-model dial behavior, the weakest grade, with bands on both axes.

On the printed frontier, the fourteen-to-twenty-one-cent shelf is contested. GPT-5.6 Luna-xhigh, GPT-5.6 Sol-low, and GPT-5.6 Luna-max interleave there; ownership flips twice within seven cents. Above that, Grok 4.5-high narrowly beats GPT-5.6 Sol-medium on both axes and owns about $0.31 to $0.45. GPT-5.6 Sol-high then owns up to $0.62, where Claude Opus 5-medium takes a narrow slice, and Sol resumes through $1.56. From there Claude Opus 5 owns the rest: xhigh to $2.03, then max.

Two of these margins are thin enough to name. GPT-5.6 Sol-max beats Claude Opus 5-high by 0.026 quality points and under 2% on cost. Grok 4.5-high beats GPT-5.6 Sol-medium by 0.238 points and half a percent. Both are real on the raw floats, and both sit inside the evaluator's own display rounding, which prints the first pair as 59 and 59 at $1.04 and $1.06. AA has already reprinted one of these costs once. Treat either relation as one reprint away from flipping.

Most models sit inside the frontier. DeepSeek holds the floor; Grok enters the middle.

• DeepSeek V4 Pro and Flash: both high/max dials print on both axes. Pro-high sits on the frontier at 43.11/$0.0405; Pro-max follows at 44.27/$0.0448. Flash-high prints 37.46/$0.0411, while its full Intelligence Index run cost was $53.94. Flash-max and Pro-high dominate it. Both measured costs fell outside my recorded estimate bands. The grading record keeps those failed estimates so later measurements can test the method.
• Claude Opus 5: its dial is low/medium/high/xhigh/max, default high, and AA scored every rung. Max and xhigh both dominate Claude Fable 5's printed point; medium takes a narrow band of its own; high and low are dominated, high by GPT-5.6 Sol-max on a razor margin and low by Sol-medium, Luna-max, and Grok-high. Its thinking-disabled mode ships at low, medium, and high only, and Anthropic's docs say requests that disable thinking at xhigh or max return an error. AA scores no row for that mode, so it stays named and off the chart.
• GPT-5.6 Terra: dominated at every setting. Claude Opus 5-medium and GPT-5.6 Sol beat it from above, while GPT-5.6 Luna, DeepSeek, and Grok 4.5 beat it from below.
• Kimi K3: its vendor dial is low/high/max. Max is the one Index-scored reasoning configuration and uses the vendor default; low and high are tier-4 estimates with bands on both axes. Max lands just under GPT-5.6 Sol-xhigh at a higher price, so it is dominated on cost and quality. It ranks fifth of 56 on combined Briefcase and still point-leads Claude Fable 5 on analytical quality with overlapping intervals, though three Claude Opus 5 rungs now sit above both.
• Grok 4.5: high is the top of its dial and its one Index-scored reasoning configuration. It knocks GPT-5.6 Sol-medium off the frontier by a quarter of a quality point and half a percent on cost, thinner than the evaluator's display rounding. A documented 0.4% AA cost reprint did not change that relation. On the judgment frontier it holds the cheap-triage rung outright, beating even Claude Opus 5 at low effort.
• Claude Sonnet 5: its max setting, 53.4 quality at $1.53, is worse than GPT-5.6 Sol-medium at nearly five times the price. Nine printed configurations beat it on both axes. Its four lower settings are projections from HLE and CursorBench effort curves with a wide error bar.
• Gemini 3.6 Flash: high is the top of its minimal/low/medium/high thinking dial and its one Index-scored configuration, printing 50.07/$0.50. Six printed configurations beat it on both axes, so it sits inside the frontier; the three lower settings are tier-4 estimates that fall deeper in. Its judgment profile splits: tenth on the Omniscience board, above Sol, but analytical quality at the floor of this comparison, above only DeepSeek V4 Flash, so the composite still ranks it low.

Effort pricing climbs as an escalator, but the shape is per-model. Along GPT-5.6 Sol's fully printed dial, each step roughly doubles the price of a marginal quality point, about three cents per point at the bottom and twenty-nine at the top, and the final step to max buys barely more than a single point. Claude Opus 5's printed dial runs the other way: $0.045, then $0.170, $0.419, and $0.748, so the steps multiply by 3.74, then 2.47, then 1.79. The expensive jump is out of low effort, and the ratio flattens toward the top. Two fully measured dials, two different shapes, so treat the doubling rule as Sol's, not a law. Either way there's no magic setting where the economics break; the question is which step your task's quality bar requires.

Claude Opus 5 takes all three work-product boards. Combined Briefcase Elo puts it at 1720.43 [1707.96–1733.73], first of 56. Analytical quality: 2015.89 [1991.24–2042.50], first. GDPval, which scores end-to-end work: 1860.57 [1835.37–1885.76], first of 180. Each of those leads clears the intervals of the model it displaced.

The old order survives intact underneath. Claude Fable 5 is fourth on combined Briefcase at 1573.78 [1562.40–1585.03] and third on GDPval at 1746.74 [1729.38–1764.11]; Kimi K3 1541.34 [1530.17–1551.50] and GPT-5.6 Sol 1504.82 [1494.03–1515.65] follow at fifth and sixth, both adjacent leads clearing the intervals. GDPval still puts Fable and Sol 10.99 points apart with overlapping intervals, and Sol is cheaper. On analytical quality Kimi's 1750.94 point-leads Fable's 1739.68 in a statistical tie, and Fable leads Sol's 1599.67 by 140.01. Sonnet sub-max has Briefcase rows but no Omniscience rows, so the composite scores only Sonnet-max. Fable's judgment rows remain max-only, while Opus 5 is measured at every effort on all three boards.

Knowledge is the one board Claude Fable 5 keeps. Omniscience ranks Fable first at 40.15, Gemini 3.1 Pro Preview second, Claude Opus 5 third at 31.27, Claude Opus 4.8 sixth, and Grok 4.5 seventh; Gemini 3.6 Flash is 10th, Sol 13th, Kimi 19th, Terra 68th, and Luna 107th. Fable's lead comes from knowing more, not from answering less. Its raw hallucination rate is 0.55, worse than Opus 4.8 at 0.36 and Kimi at 0.51; Opus 5 sits near 0.50 at every effort. Sol is the extreme at 0.89. METR also reports a high detected-cheating rate for Sol on one ReAct harness, though METR calls the numbers non-robust.

One neutral harness, two separate axes. Artificial Analysis runs Intelligence Index v4.1 in uniform scaffolding across vendors. Quality is the Index score. Cost is its benchmark-weighted model bill per Intelligence-Index task run, based on token use and token prices. It is not conditioned on success and includes neither retries nor human time. Per-token price can still mislead because models consume different amounts of input, cached, reasoning, and answer tokens on the same task; per-task cost captures that token appetite, while the quality axis captures performance. Claude Fable 5's printed point is the evaluator's adaptive-reasoning configuration with automatic Claude Opus 4.8 fallback.

Why not just read each vendor's own benchmarks? Because the harness moves scores as much as the model does. The same Claude Opus 4.5 swings 9.5 points on SWE-bench Pro between a standardized harness and a vendor one. A cross-vendor gap measured under different harnesses is noise wearing a number, so only one neutral evaluator's runs compare.

“Stop comparing LLM agents without disclosing the harness.”

Source: arXiv preprint 2605.23950 (May 2026), source of the 9.5-point same-model swing

Every point sits at the best evidence grade it can honestly hold: printed on both axes; projected from vendor effort curves; evaluator-flagged quality with cost inferred from measured dials; or, weakest, a tier-4 cross-model estimate. Bands show measured donor spread or recipe sensitivity. They do not represent full confidence intervals. Tier names are never matched across vendors, and a setting a vendor doesn't ship is never invented. The printed only toggle strips everything below the top grade.

The judgment frontier is a composite. It blends Briefcase analytical-quality Elo with the Omniscience knowledge-reliability index. Each component is scaled across the compared set and blended half and half. Sweeping the blend weight from one quarter to three quarters forms the band; cost comes from Briefcase's printed workload bills. Use it for ordering and rungs. Every dominance verdict is checked against the weight sweep, Elo confidence intervals, and dropping any single model.

Other priced configurations. This comparison covers twelve chosen models. Artificial Analysis lists about ninety more priced configurations outside that scope. Examples include Gemini 3.5 Flash, GPT-5.5 at xhigh, and GLM-5.2 at max. Each is dominated by a charted printed configuration. The rest are not surveyed or judged here.

The evidence limits change how you read the dots. Claude Fable 5, Claude Opus 4.8, and Claude Sonnet 5 below max are projections. Kimi K3, Grok 4.5, and Gemini 3.6 Flash sub-max are weaker tier-4 estimates. No row uses the cost-estimated class. Per-task costs are benchmark-sized and scale with your own tasks. Values and evidence grades can move within weeks. The judgment frontier also blends two evaluator benchmarks into a score of my own design. Use it to order the options. The finer print:

• Claude Opus 5's four sub-max rungs rest on a single capture. AA published them during the day I worked the model up: an archive snapshot of the same page two hours earlier had only max scored, xhigh and high present as bare directory entries, and no medium or low row at all. Max is verified across two capture times; the other four are corroborated across seven payloads fetched at the same moment, which is agreement between sources rather than over time. A model scored on its release day is exactly where an evaluator reprint is most likely, and the headline claims run through those rungs.
• Claude Fable 5, Claude Opus 4.8, and Claude Sonnet 5 below max are projections, never neutral Index measurements. Fable-medium is 58.44 ±1.3 / ~$0.8323, with roughly ±40% cost-shape uncertainty; the judgment benchmarks score Fable-max only.
• Claude Fable 5's printed numbers include its automatic fallback to Claude Opus 4.8 on a share of tasks; the evaluator plots the with-fallback configuration, and this post uses it consistently.
• The Claude Opus 4.8 dominance claim is solid on printed values at its measured configuration and applies only at point estimates below it.
• Projected costs borrow GPT-5.6 Sol's dial shape, so cross-model shape uncertainty remains. Claude Opus 5 supplies the first out-of-sample test of that borrowing: against its real printed cost dial, Sol's shape errs by 15%, 16%, 1%, and 7% at xhigh, high, medium, and low. That is inside the stated ±40%, but it tests the assumption on one model, not the projections themselves, which are for different models and stay ungraded.
• Adding Claude Opus 5's dial to the tier-4 donor pool superseded every tier-4 estimate on this chart. Three of the seven midpoints moved outside the band the previous donor set published. That is why no tier-4 number should be read without its band.
• The chart covers GPT-5.6 standard-mode low through max. It excludes none under AA's non-reasoning class and the orthogonal pro mode, for which AA has no rows. OpenAI's model-specific guide does not list legacy minimal; whether the API rejects or aliases it is untested.
• Speed isn't on the chart and matters: GPT-5.6 Luna's printed token rate runs about three times GPT-5.6 Sol's.
• Judgment coverage varies: Briefcase prints no rows for GPT-5.6 Luna or GPT-5.6 Terra; GDPval prints both. Sonnet sub-max has Briefcase rows but no Omniscience rows. Fable judgment evidence remains max-only, while Claude Opus 5 is scored at every effort on all three boards. Luna's knowledge index is negative across its whole dial. Kimi K3, Grok 4.5, Sonnet 5, and Gemini 3.6 Flash each have one Intelligence-Index-scored reasoning configuration.
• AA values and evidence grades are mutable. Dated captures document Terra and Grok cost reprints, two DeepSeek estimates replaced by measurements, and a same-time disagreement between Sol's evaluation-board and model-page values.
• The analytical-quality gap comes from one evaluator's Elo panel and rubric scores. METR's cheating figure comes from one public-model harness and is non-robust by METR's own description.
• The judgment frontier is my own composite of two evaluator boards. Min-max scaling makes it set-relative; checks cover the weight sweep, Elo intervals, and set membership. Its cost comes from the judgment workload rather than the main chart.
• Claude Fable 5 spent most of June suspended under a US export-control directive before global access returned in July. Availability can move as fast as scores.