Which model, at which effort?

The rail measures a whole job, not one benchmark task. Each dot is a model and effort that no cheaper measured configuration beats on the profile score. A dot sits at what that rung contributes to the job: its lane's share of the job, times what the rung costs against the best rung in its own lane. The right end of the rail is the job with both lanes at their best, so the ticks read 1, .1, .01 — fractions of that job, not dollars. Equal distances still mean equal cost ratios.

Why not one dollar figure for both lanes? Because a benchmark task is not a job, and the two lanes' tasks are not the same size. A Figure task is a set of hard questions and a business analysis, scored per question. An Execute task is a repository issue or a terminal session an agent works for minutes. Charging both lanes the same dollar ceiling treats one of each as an equal purchase — which it is not, when a real job runs one lane once and the other for hours.

"Your job is" sets the split. Mostly figuring puts 20% of the job in Execute, Balanced puts half, Mostly executing puts 80% there. The line is then one threshold both lanes' contributions sit under, and it converts back to a dollar ceiling per lane before anything is chosen — the lane carrying most of the job gets squeezed first, which is the point. Each answer still shows its own benchmark dollars per task and the date of the capture it came from.

Those three numbers are chosen, not measured. They are round positions, from the author's own working shape averaged informally with colleagues'. There is no retained source behind 20, 50 and 80, and none is claimed: they exist so a reader can say roughly what their week looks like, and a spinner would promise a precision nobody here has. Pick the one nearest your own and read the answer it gives.

The author's own jobs are a separate thing, and they are measured. Across 16 of his Claude Code jobs that ran both a figuring phase and an executing phase, 2026-09-03 to 09-08, executing took 5 to 94 percent of those two phases' API-equivalent cost at that day's published rates — median 84 percent, 75 percent weighted by dollars, over $347 of attributed spend. The largest such job spent $63 of $77 executing. That is one person's five days on one harness. It is not a bill, it is not a measurement of your work, and the width of that range is exactly why the split is a control rather than a constant.

Those runs were also almost all cache reads: 98.4 percent of input-side tokens across the Claude Code sessions, and 96.1 percent of input tokens across 258 Codex sessions in the same window. For work shaped like that a model's cache-read price orders the bill far more than its input price does, and a benchmark task bounded at a few minutes cannot show you that. Opus 5 reads cache at $0.50 per million tokens, Fable 5.1 at $0.25, Sonnet 5 at $0.20, Haiku 4.5 at $0.10 (Anthropic pricing, read 2026-09-08).

The rail carries the undominated rungs only. Cheaper and stronger steps are the dots themselves: every measured Pareto step for both lanes is on the line, and dragging to one selects it. A ceiling below every rung produces no recommendation for that lane and names the cheapest measured option instead.

A model does not need every benchmark to earn consideration. One strong, relevant evaluation can qualify a scoped alternative. Figure looks across practical analysis, reasoning, factual reliability and document work. Execute looks across coding-agent work, professional outputs and tool workflows. A specialist remains a specialist; its best result does not establish broad capability.

Within your available models and effort settings, each benchmark admits results within 5 score points of its leader, or 100 Elo on native Elo boards. It also admits undominated quality–cost options within 20 points, or 400 Elo, of the leader. “Undominated” means no measured option is both at least as good and at least as cheap, with one strict advantage. Missing cost cannot earn a value claim. These margins are chosen screening rules, not uncertainty estimates or proof of a meaningful lead. Cheaper results below the quality floor remain in the dated discovery survey for follow-up.

The selected focus shows its core benchmarks and relevant supporting evidence. Coding agent work also shows FrontierCode, Zapier, AA's own Terminal-Bench run and SciCode; Deliverables shows Briefcase rubric checks and agentic professional work; Codebase investigation shows long-context reasoning and document reading. Supporting scores do not enter the profile average. Every qualifying model remains accessible; the first six are a compact alphabetical preview. Open a model’s row to see the exact efforts, benchmark scores, source, date, harness and fallback status. Several strong results from one publisher are not independent corroboration. Different effort settings on one card are separate observations, not a combined configuration.

The shortlist uses native Briefcase and GDPval Elo and the published AA-Omniscience index. Composite profiles use the fixed conversions below; the native evidence stays visible beside them. Native and composite units must not be compared directly. FrontierCode Fable runs have undisclosed fallback settings; Kimi's Zapier effort is undisclosed. These results can support scoped consideration, but cannot fill an exact broad-composite gap.

Start with quality, then decide how much to spend. Quality first selects the highest measured profile score among configurations with all required quality evidence, comparable API costs and reproducible settings. Equal scores prefer lower cost, then a stable configuration identity. It does not exchange score points for a guessed dollar value, and it does not claim to identify the best unmeasured model.

Choose Figure it out or Execute it, then narrow the work when a focused profile fits. A deliverables result can qualify on its own evidence even when the Coding Agent Index has not run that configuration and so cannot give it a broad Execute score. Undisclosed effort or fallback settings remain scoped observations; they cannot set the recommendation.

The rail sets a logarithmic ceiling in shares of one job: equal distances mean equal cost ratios, and the position converts to a dollar ceiling per lane through that lane's share of the job. Each kind of work then answers with its highest score within its own ceiling. Every measured Pareto step is a dot on the rail, so dragging past one selects it. A ceiling too low produces an explicit empty recommendation with the cheapest available alternative. Releasing the rail at its top removes the ceiling — Quality first — without changing model or effort exclusions.

Cheaper models that are not on the frontier still appear where their evidence earns it: the shortlist under Why this choice states each qualifying model’s measured scope, system, and unknown or above-ceiling costs. A component strength is not a broad recommendation.

The audit tests the policy, not the probability of being right. Open Method sensitivity & winning margins to compare half/double Elo slopes, wider group weights, omitted benchmarks/groups and focused compositions. Fixed cohorts keep the baseline configurations; expanded cohorts separately admit newly scoreable ones. Quality-only results allow unknown costs; priced results require comparable costs and apply your ceiling. Your access filters apply throughout. Scenario counts are not confidence percentages.

The reproducible all-model audit retains winners, coverage and native component margins. A stable result can still be wrong for your task: the scenarios cannot supply representative real-world outcomes that were never measured.

Practical guidance for each kind of work, from the configuration the rail names. There is no measured quota optimum.

Benchmark API cost is a comparison unit, not your bill. Subscription plans may include an allowance, require credits from the first request, restrict supported tools or leave exact versions undisclosed. The selected model’s access disclosure and the full access reference keep those distinctions separate, with official sources and verification dates.

Downloadable weights establish a self-hosting route, not laptop suitability or savings. The reference gives sourced serving examples and distinguishes author formats, community quantizations and related hosted variants. For example, hosted Qwen Flash is based on Flash-Next with additional production features; it is not silently treated as the same measured weights configuration. Local hardware, electricity and operations costs are not estimated here.

Turning the dial up buys the model more thinking. You pay for that thinking in tokens, so it is a spending dial first and a quality dial second.

That makes it measurable. Every row here is one model at one setting, scored on DeepSWE, Terminal-Bench 2.1 and SWE-Atlas-QnA and billed with the same job weights. The ladder compares complete measured settings; a missing setting is skipped, not estimated.

GPT-5.6 Sol shows it most starkly. Among models with at least four complete settings, the last step costs about 44× what the first did: $0.017 a point at the bottom, $0.74 at the top. Measured September 6, 2026.

No number here tells you where to stop. The board prices a point at each step. What a wrong answer costs you is the other half, and only you have it.

Higher does not always mean better, either. 2 measured configurations score lower at a higher setting:

• Claude Opus 5 · xhigh → max — 1.12 points down, $0.77 more per task. Paying more for a worse result.
• GPT-5.6 Sol · high → xhigh — 0.78 points down, $0.74 more per task. Paying more for a worse result.

One board, one capture date, no error bars. Read that as a reason to check your own setting, not as a law about the model. Either way, “turn it to max” is a guess rather than advice.

Never pick the setting on its own. A cheap model turned up and a costly one turned down are the same kind of thing here — which is why the card compares configurations rather than ranking model names.

A profile defines the work being compared. Four options, two per lane, each scored by one named source — so a focus can never quietly mix in evidence its own definition does not name. There used to be eight, which asked you which benchmark you trusted rather than what you were about to do.

What scores the Execute lane

The Execute lane reads the Artificial Analysis Coding Agent Index, captured at v1.4 on 2026-09-08, whose own page states it "incorporates 3 benchmarks: DeepSWE, Terminal-Bench v2.1, and SWE-Atlas-QnA" in equal thirds, over three attempts per task.

A row on that board is a harness, a model and a reasoning setting together — "Claude Code / Fable 5.1 / max", not "Fable 5.1". That is the whole reason the lane moved to it: the number describes the pair you actually run, not a model in a scaffold nobody uses. Choosing Claude Code reads Claude Code's rows; choosing Codex reads Codex's; All models takes each configuration's best published row whatever agent produced it, and the answer names that agent. Cost is that row's own mean API dollars per task, cached input and cache writes included.

Its three parts: DeepSWE resolves repository issues end to end (113 tasks); Terminal-Bench v2.1 runs command-line coding, administration and data tasks (89 tasks); SWE-Atlas-QnA answers expert-written comprehension questions inside a repository (124 tasks). The last of those, alone and in percent, is what Codebase investigation scores on the Figure lane.

HLE tests hard academic questions; AA-LCR tests reasoning across document sets. Omniscience supplies the publisher's factual-reliability score. Briefcase separates analytical quality, rubric checks and presentation; its presentation score is never substituted for analysis. GDPval-AA v2 compares professional deliverables and is what Deliverables scores, alone, in its native Elo. FrontierCode Main and Zapier AutomationBench are now supporting evidence on the rows that carry them, not requirements: neither gates a lane any more.

Fixed conversions preserve differences without depending on the current roster. Composite profiles use:

Rate p                     → 100 × p
Published Omniscience O     → (O + 100) / 2
Native Briefcase/GDPval E   → (E − 500) / 20

Elo is not clipped: 1000 becomes 25 points, 1500 becomes 50 and 2500 becomes 100; higher or negative values remain possible. Twenty Elo per point is a declared cross-benchmark policy, not an empirically established exchange rate. 1000 Elo is not asserted to mean human-level performance. Single-benchmark profiles keep their native units. Adding or filtering models cannot move another configuration's score.

Costs use the same profile weights as quality. AA bills use per-benchmark input/output tokens, cache rates, prices and fallback fractions, divided by task count. Both Briefcase dimensions use its full task bill. The Coding Agent Index publishes its own mean API dollars per task per row, already inclusive of cached input and cache writes. FrontierCode publishes mean API dollars per rollout, already per task. Zapier publishes API dollars per task rounded to cents. Dedicated-deployment prices and incomplete fallback bills remain unpriced. Subscription inclusion is never substituted with zero benchmark cost.

Each profile requires all its components at the exact model, effort, version, harness and compatible system. Missing values stay unscored or unpriced; weights are never renormalized around them. A Coding Agent Index row is used only for the harness that produced it — a Codex number never wears a Claude Code label. A fallback-enabled AA system cannot borrow a standalone external result, even when observed fallback use is zero; the index's own fallback-enabled rows stay eligible and say so, the same rule the Figure lane applies to AA's. Focused source joins preserve model, effort, harness, fallback identity and source record. An estimated overall AA Index neither fills a gap nor excludes complete evidence.

What this composition costs you

Coverage. The index publishes 68 rows, of which 53 name a model compared here; they cover 49 model-and-effort configurations as of 2026-09-08 — 17 reachable under Claude Code and 21 under Codex once model compatibility is applied. A configuration the board has not run carries no Execute score, with that reason attached, and stays visible in the shortlist. Claude Fable 5.1 is published only at max, in Claude Code; GPT-6 Astra only at max, in Codex — so a reader running either below max is answered from the shortlist instead. The composition this one replaced required FrontierCode, Briefcase rubric, GDPval and Zapier all present, which admitted 29 configurations and excluded every Fable row — the reader's own model missing from the lane that decides their biggest bill.

The demoted sources still agree with it. Over the 31 configurations both measure, FrontierCode ranks them at Spearman 0.90 against the index; Zapier, over 28, at 0.89. Requiring FrontierCode was buying agreement the index already had, at the price of every configuration FrontierCode does not publish. Both remain on the rows.

Where the ordering comes from. Inside the index, Terminal-Bench 2.1 sits near its ceiling for the leading rows: across the ten highest-scoring rows in the 2026-09-08 capture it averages 86.6 against a board maximum of 91.0, while DeepSWE averages 65.0 and SWE-Atlas-QnA 49.2. The top of the board is therefore separated mostly by DeepSWE and SWE-Atlas-QnA, not by the terminal component.

The board moves. Between May and August 2026 the index went through four versions. Its benchmark line-up changed once, at v1.1 in June 2026, which added DeepSWE and removed SWE-Bench-Pro-Hard-AA. v1.2 and v1.3 changed how SWE-Atlas-QnA is scored — rubric reward, then binary pass/fail, then Scale's own Task Resolve Rate — and v1.4 upgraded Terminal-Bench from v2 to v2.1 and added reward-hacking detection (AA methodology, read 2026-09-08). Scores are not comparable across those versions. The extractor asserts the published version sentence, the exact set of dataset slugs and the equal-third weights, and stops the refresh rather than re-weighting the lane underneath you if any of them changes.

Its two coding parts each carry a caveat. DeepSWE (Datacurve, July 2026) is 113 original tasks over 91 repositories, graded binary with no partial credit, run for every model in one fixed harness — a deliberate trade its own paper describes as removing the scaffolding confound at the cost of cross-harness realism. It is also a single-vendor benchmark whose contamination controls cannot be audited from outside. SWE-Atlas-QnA (Scale AI, March 2026) is 124 questions over 11 production repositories; Scale's own leaderboard notes some configurations show an elevated refusal rate because the tasks trip security filters, which reads as a low score without being one.

Harness effects are the point, and also a limit. The number describes one agent at one version, and it does not transfer to the same model in another scaffold. That is not a small correction: a controlled 3×3 study on a 100-task SWE-bench Verified subset measured harness variance at 7.8× model variance and found rank reversals in 6 of 9 model-pair comparisons (arXiv 2605.23950, a position paper whose authors explicitly decline to claim that ratio is universal), and Harness-Bench found a 23.8-point aggregate gap between the best and worst configurable harnesses over the same 106 tasks and model pool (arXiv 2605.27922). Both were read 2026-09-08. This is the argument for scoring the pair you run — and the reason a number from one agent tells you little about another.

What the Figure lane's own sources are worth

The Execute lane is not the only one holding evidence with a record. Stated plainly, and dated:

• HLE has contested answers. FutureHouse examined 321 text-only chemistry and biology questions and found that 29% ± 3.7 had answers contradicted by peer-reviewed literature (FutureHouse, 23 July 2025). The HLE team's own follow-up on that page put roughly 18% of a reviewed subset as problematic under three-expert review and committed to rolling revisions. The finding covers that subset, not the whole exam.
• AA-Omniscience's questions are machine-written. Artificial Analysis states its 6,000 questions were generated by an LLM-based question-generation agent from authoritative academic and industry sources (AA, read 2026-09-08). The sources are human; the questions are not.
• AA-LCR v1.1 corrected its own answers. AA's methodology records that v1.1 added a grading system prompt, corrected 16 answer keys and changed the grader, and that scores are not directly comparable with v1.0 (AA methodology, read 2026-09-08).
• Briefcase is Artificial Analysis's own private benchmark. Its 91 tasks across four held-out scenarios were written by outside experts from firms including Google, McKinsey and BCG, but AA owns and runs it, only a fifth demonstration scenario is public, and AA's methodology table lists one repeat per task — no averaging over attempts (AA-Briefcase, read 2026-09-08). Its analytical-quality Elo is a preference ranking, which is not a correctness measure.

These are benchmark proxies, not success probabilities. Collaborative problem framing and planning remain unmeasured in Figure; selective escalation and supervision remain unmeasured in Execute. Agent harnesses, tools, task sets and judges differ. Shared AA provenance is not independent corroboration, and small numerical gaps need not matter in real work. A one-benchmark profile carries that scope next to its recommendation.