Depends what the job is worth. Drag the line

Your job

Figure it outplanning & research

Claude Opus 5.5 max≈$9.27 / task · 79.3 pts

Execute itcode & commands

Claude Sonnet 5.5 max≈$7.37 / task · ≈68.8 ptsClaude Sonnet 5.5 max≈68.865.9–71.6Claude Opus 5.5 max≈67.264.4–69.9The ranges overlap, so either could lead.

past redGPT-6.1 Sol low + MiMo-V2.6-Flash reasoning26× cheaper
past amberGPT-6.1 Sol high + GPT-6.1 Sol low17× cheaper

Execute cost includes an accounting scenario and a separately predicted missing benchmark; the actual total bill and its range are unknown.

Every configuration

quality against cost · open ↓close ↑

Higher is better, left is cheaper. The dashed line joins the rungs on the rail above; the ring is the pick.

Figure score against cost per task, 109 configurations, log cost axis. Higher is better; left is cheaper.Scored on Briefcase analytical quality, AA-LCR 1.1, AA-Omniscience index; the flags mark the pick.

Conditional tier envelope $0.019–$0.093 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.044–$0.22 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.072–$0.36 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.11–$0.57 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.21–$1.05 per mix task; midpoint scenario, not expected price.

Claude Opus 5.5 · maxthe pick · ≈$9.27 / task · 79.32 pts

Chart key & sources

OpenAI

GPT-6.1 SolGPT-6 LunaGPT-6 SolGPT-6 AstraGPT-5.6 LunaGPT-5.6 SolGPT-5.6 TerraGPT-5.5

Anthropic

Haiku 5.5Sonnet 5.5Opus 5.5Fable 5.1Opus 5Fable 5

Moonshot

Kimi K3

DeepSeek

DeepSeek V4.1 FlashDeepSeek V4 Flash VisionDeepSeek V4 Pro 0813

xAI

Grok 4.7Grok 4.6

Google

4 Argon3.8 Flash3.7 Flash3.5 Flash

Zhipu

GLM-5.3-FlashGLM-5.3

Alibaba

Qwen3.8-Flash-NextQwen3.8 27B

Models you can use

44 of 44 included · open ↓close ↑

Turn off what you can’t use; the rail recomputes. Open a model to drop single efforts.

5/5 efforts · edit

6/6 efforts · edit

7/7 efforts · edit

5/5 efforts · edit

1/1 effort · edit

5/5 efforts · edit

3/3 efforts · edit

2/2 efforts · edit

1/1 effort · edit

3/3 efforts · edit

4/4 efforts · edit

3/3 efforts · edit

2/2 efforts · edit

1/1 effort · edit

2/2 efforts · edit

1/1 effort · edit

4/4 efforts · edit

1/1 effort · edit

2/2 efforts · edit

1/1 effort · edit

2/2 efforts · edit

1/1 effort · edit

3/3 efforts · edit

1/1 effort · edit

How this is measured

sources, weights and limits · open ↓close ↑

How the rail is builtundominated rungs on a log job-share axis

The rail measures a whole job, not one benchmark task. Each dot is a model and effort that no cheaper comparable configuration beats on the mix score. A dot sits at what that rung contributes to the job: its lane's share of the job, times what the rung costs against the best rung in its own lane. The right end of the rail is the job with both lanes at their best, so the ticks read 1, .1, .01 — fractions of that job, not dollars. Equal distances still mean equal cost ratios.

Why not one dollar figure for both lanes? Because a benchmark task is not a job, and the two lanes' tasks are not the same size. Figure combines business analysis, document reasoning and factual reliability. Execute combines repository work, terminal tasks, professional deliverables and app workflows. Their benchmark task sizes differ. Charging both lanes the same dollar ceiling treats one of each as an equal purchase — which it is not, when a real job runs one lane once and the other for hours.

"Your job is" sets the split. Mostly figuring puts 10% of the job in Execute, Balanced puts half, Mostly executing puts 90% there. The line is then one threshold both lanes' contributions sit under, and it converts back to a dollar ceiling per lane before anything is chosen — the lane carrying most of the job gets squeezed first, which is the point. Each answer still shows its own benchmark dollars per task and the date of the capture it came from.

Those three numbers are chosen, not measured. They are round positions, with the two "mostly" options symmetric around Balanced. There is no retained source behind 10, 50 and 90, and none is claimed: they exist so a reader can say roughly what their week looks like, and a spinner would promise a precision nobody here has. Pick the one nearest your own and read the answer it gives.

The author's own jobs are a separate thing, and they are measured. Across 16 of his Claude Code jobs that ran both a figuring phase and an executing phase, 2026-09-03 to 09-08, executing took 5 to 94 percent of those two phases' API-equivalent cost at that day's published rates — median 84 percent, 75 percent weighted by dollars, over $347 of attributed spend. The largest such job spent $63 of $77 executing. That is one person's five days on one harness. It is not a bill, it is not a measurement of your work, and the width of that range is exactly why the split is a control rather than a constant.

Those runs were also almost all cache reads: 98.4 percent of input-side tokens across the Claude Code sessions, and 96.1 percent of input tokens across 258 Codex sessions in the same window. For work shaped like that a model's cache-read price orders the bill far more than its input price does, and a benchmark task bounded at a few minutes cannot show you that. Opus 5 reads cache at $0.50 per million tokens, Fable 5.1 at $0.25, Sonnet 5 at $0.20, Haiku 4.5 at $0.10 (Anthropic pricing, read 2026-09-08).

The rail carries the undominated rungs only. Cheaper and stronger steps are the dots themselves: every Pareto step for both lanes is on the line, and dragging to one selects it. A ceiling below every rung produces no recommendation for that lane and names the cheapest comparable option instead.

How alternatives qualifyscreening rules and evidence limits

A model does not need every benchmark to earn consideration. One strong, relevant evaluation can qualify a scoped alternative. Figure looks across practical analysis, reasoning, factual reliability and document work. Execute looks across coding-agent work, professional outputs and tool workflows. A specialist remains a specialist; its best result does not establish broad capability.

Within your available models and effort settings, each benchmark admits results within 5 score points of its leader, or 100 Elo on native Elo boards. It also admits undominated quality–cost options within 20 points, or 400 Elo, of the leader. “Undominated” means no measured option is both at least as good and at least as cheap, with one strict advantage. Missing cost cannot earn a value claim. These margins are chosen screening rules, not uncertainty estimates or proof of a meaningful lead. Cheaper results below the quality floor remain in the dated discovery survey for follow-up.

Each track shows its mix and relevant supporting evidence. Figure also carries hard-question and document-reading results; Execute carries FrontierCode, Zapier, SciCode, Briefcase rubric checks and other agent-work results. Supporting scores do not enter the mix. Every qualifying model remains accessible; the first six are a compact alphabetical preview. Open a model's row to see efforts, native scores, source, date, harness and fallback status. Several results from one publisher are not independent corroboration.

The shortlist uses native Briefcase and GDPval Elo and the published AA-Omniscience index. Both mixes use the fixed conversions below; the native evidence stays visible beside them. Native and composite units must not be compared directly. FrontierCode does not say whether Claude Code rerouted a Claude run to a fallback model. Its Claude results count only for models Claude Code never reroutes. Kimi's Zapier effort is undisclosed. Such scoped results can support consideration, but cannot fill an exact mix gap.

How the recommendation is pickedquality, spending and uncertainty

Start with quality, then decide how much to spend. Quality first selects the highest score in the declared mix among configurations with reproducible settings and enough measured or explicitly estimated inputs. Equal scores prefer lower cost, then a stable configuration identity. Estimates can keep useful models in the comparison, but when the pick or an alternative under the same ceiling is estimated and their ranges overlap, the answer is marked close call, which opens a note naming both configurations. The range does not include all uncertainty in the measured benchmarks.

There is one mix for Figure it out and one for Execute it. The provider chips filter models you can access: Codex is OpenAI-only; Claude Code is Anthropic-only. Each benchmark keeps the harness that measured it. The composite does not claim to measure the model inside either product.

The rail sets a logarithmic ceiling in shares of one job: equal distances mean equal cost ratios, and the position converts to a dollar ceiling per lane through that lane's share of the job. Each kind of work then answers with its highest score within its own ceiling. Every Pareto step is a dot on the rail, so dragging past one selects it. A ceiling too low produces an explicit empty recommendation with the cheapest available alternative. Releasing the rail at its top removes the ceiling — Quality first — without changing model or effort exclusions.

The floors are the author's judgment, not a measurement. Below 60 points for Figure it out, or 40 for Execute it, he would not run that lane at all; from there up to 70 and 50 a configuration is acceptable but not strong. The rail and the Every configuration chart shade those stretches red and amber, and an answer that lands in one says so. The floors never change the pick — it is still the highest score under your ceiling — and no source stands behind the numbers.

Cheaper models that are not on the frontier still appear where their evidence earns it: the shortlist under Why this choice states each qualifying model’s measured scope, system, and unknown or above-ceiling costs. A component strength is not a broad recommendation.

The audit tests the policy, not the probability of being right. Open Method sensitivity & winning margins to compare half/double Elo slopes, wider group weights, omitted benchmarks/groups. Fixed cohorts keep the baseline configurations; expanded cohorts separately admit rows that meet the measured score threshold under each scenario's weights. Quality-only results allow unknown costs; priced results require comparable costs and apply your ceiling. Validated estimates stay fixed during these weight changes; this audit tests weighting policy rather than prediction error. Your access filters apply throughout. Scenario counts are not confidence percentages.

The reproducible all-model audit retains winners, coverage and native component margins. A stable result can still be wrong for your task: the scenarios cannot supply representative real-world outcomes that were never measured. The Execute leader and its limits explains what the Execute pick with no budget limit rests on, for All models and Claude Code.

Why this choicemeasurements, method & access

109 of 121 configurations can set this lane’s recommendation, 21 of them with estimates.

Why 12 configurations cannot set this recommendation

GPT-5.4 Pro xhigh · Benchmark-specific harnesses
Score unavailable · Comparable cost unavailable
Only 0% of score weight measured; at least 50% required for a recommendation
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Token accounting is not published for this benchmark
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Token accounting is not published for this benchmark
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Token accounting is not published for this benchmark
GPT-5.5 Pro xhigh · Benchmark-specific harnesses
Score unavailable · Comparable cost unavailable
Only 0% of score weight measured; at least 50% required for a recommendation
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Token accounting is not published for this benchmark
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Token accounting is not published for this benchmark
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Token accounting is not published for this benchmark
Step 5 Preview unspecified · Benchmark-specific harnesses
65.41 points · $0.75 / profile task
Reasoning effort is not disclosed; cannot reproduce an exact setting
Qwen3.8 Max 0902 unspecified · Benchmark-specific harnesses
68.29 points · $5.95 / profile task
Reasoning effort is not disclosed; cannot reproduce an exact setting
Mistral Large 4 Preview unspecified · Benchmark-specific harnesses
58.49 points · $0.76 / profile task
Reasoning effort is not disclosed; cannot reproduce an exact setting
Ling 3.0 Flash VL unspecified · Benchmark-specific harnesses
48.63 points · Comparable cost unavailable
Reasoning effort is not disclosed; cannot reproduce an exact setting
- Briefcase analytical quality: Comparable API pricing is not published
- AA-LCR 1.1: Comparable API pricing is not published
- AA-Omniscience index: Comparable API pricing is not published
Claude Fable 5 high · Benchmark-specific harnesses
Score unavailable · ≈$3.52 / profile task (estimated)
Fallback behavior is not disclosed; exact system cannot be established
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
How the estimate is made
0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$3.52 per mix task; range $1.81–$7.11.
Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.
- Briefcase analytical quality · score: 48.8 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- Briefcase analytical quality · cost: $11.16 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · score: 82.0 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · cost: $0.51 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-Omniscience index · cost: $0.053 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
Claude Fable 5 low · Benchmark-specific harnesses
Score unavailable · ≈$1.22 / profile task (estimated)
Fallback behavior is not disclosed; exact system cannot be established
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
How the estimate is made
0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$1.22 per mix task; range $0.39–$3.89.
Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.
- Briefcase analytical quality · score: 41.6 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- Briefcase analytical quality · cost: $3.79 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · score: 80.4 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-Omniscience index · cost: $0.017 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
Claude Fable 5 medium · Benchmark-specific harnesses
Score unavailable · ≈$2.16 / profile task (estimated)
Fallback behavior is not disclosed; exact system cannot be established
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
How the estimate is made
0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$2.16 per mix task; range $1.10–$4.39.
Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.
- Briefcase analytical quality · score: 46.2 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- Briefcase analytical quality · cost: $6.79 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · score: 81.4 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · cost: $0.37 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-Omniscience index · cost: $0.031 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
Claude Fable 5 xhigh · Benchmark-specific harnesses
Score unavailable · ≈$4.77 / profile task (estimated)
Fallback behavior is not disclosed; exact system cannot be established
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
How the estimate is made
0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$4.77 per mix task; range $2.11–$11.12.
Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.
- Briefcase analytical quality · score: 49.9 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- Briefcase analytical quality · cost: $15.17 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98.
  The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · score: 82.2 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · cost: $0.63 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.
  The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-Omniscience index · cost: $0.073 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16.
  The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
Gemini 3.1 Pro Preview high · Benchmark-specific harnesses
Score unavailable · ≈$0.63 / profile task (estimated)
Only 0% of score weight measured; at least 50% required for a recommendation
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
How the estimate is made
0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$0.63 per mix task; range $0.079–$4.96.
Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.
- Briefcase analytical quality · score: 4.7 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52.
  The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- Briefcase analytical quality · cost: $1.92 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · score: 72.3 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · cost: $0.16 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-Omniscience index · cost: $0.0085 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
Qwen3.8 Max xhigh · Benchmark-specific harnesses
Score unavailable · ≈$1.21 / profile task (estimated)
Only 0% of score weight measured; at least 50% required for a recommendation
- Briefcase analytical quality: No published measurement at this exact configuration
- Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
- AA-LCR 1.1: No published measurement at this exact configuration
- AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
- AA-Omniscience index: No published measurement at this exact configuration
- AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
How the estimate is made
0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$1.21 per mix task; range $0.39–$3.85.
Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.
- Briefcase analytical quality · score: 40.0 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- Briefcase analytical quality · cost: $3.76 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · score: 80.1 converted points, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.
  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.
- AA-Omniscience index · cost: $0.017 per benchmark task, estimated.
  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.
  Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16.
  Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Evidence behind the chosen configuration

The weights are declared policy, not a validated prediction of your work. Collaborative framing and planning remain unmeasured.

Score & native measurements

79.32 points · Figure it out. Benchmark proxies do not guarantee correctness.

Benchmark	Native result	API $ / task
Briefcase analytical quality30.0% weight · AA benchmark harnessClaude Opus 5.5 (Max, Default Fallback)	2142.71 Elo	≈$29.90 (estimated)
AA-LCR 1.130.0% weight · AA benchmark harnessClaude Opus 5.5 (Max, Default Fallback)	84.67%	≈$0.76 (estimated)
AA-Omniscience index40.0% weight · AA benchmark harnessClaude Opus 5.5 (Max, Default Fallback)	46.42 points	≈$0.18 (estimated)

Supporting evidence, outside the score: Briefcase presentation 1714.96 Elo · 2026-10-18. Same exact model and effort; AA benchmark harness.

How the estimate is made

100% of score weight measured. Score 79.3; no score component estimated. API cost ≈$9.27 per mix task; range $6.58–$11.96.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

Briefcase analytical quality · cost: $29.90 per benchmark task, estimated.
Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.
Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.
AA-LCR 1.1 · cost: $0.76 per benchmark task, estimated.
Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.
Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.
AA-Omniscience index · cost: $0.18 per benchmark task, estimated.
Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.
Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Claude Haiku 5.5 max Figure: $0.21–$1.05 per mix task, a conditional tier envelope. The rail uses its $0.63 midpoint scenario, not an expected price.

At this ceiling: lower-tier scenario → Claude Opus 5.5 max; midpoint → Claude Opus 5.5 max; upper-tier scenario → Claude Opus 5.5 max. The budget choice stays the same at these three scenarios.

Tier assumptions and all measured efforts

Only request-tier allocation varies, using captured AA tokens, cache-hit rate, five-minute cache writes and task counts. Every request above 100,000 prompt tokens, including cached tokens, costs five times as much. This envelope does not guarantee an actual bill or establish a confidence interval. Other billing modifiers are outside it. Estimates may exceed your ceiling. All tier rows move together in these three readings; they do not cover every independent allocation.

Claude Haiku 5.5 low: $0.019–$0.093 per Figure mix task; 53.1 points measured.
Claude Haiku 5.5 medium: $0.044–$0.22 per Figure mix task; 60.5 points measured.
Claude Haiku 5.5 high: $0.072–$0.36 per Figure mix task; 62.3 points measured.
Claude Haiku 5.5 xhigh: $0.11–$0.57 per Figure mix task; 64.4 points measured.
Claude Haiku 5.5 max: $0.21–$1.05 per Figure mix task; 68.0 points measured.

Execute predicts its missing DeepSWE component separately; its total bill has no established bound.

Access: subscription, API or your hardware

Reviewed 2026-10-08. Availability does not reproduce a benchmark’s agent, effort or fallback configuration.

Subscription Verified route

Available in Claude and Claude Code paid plans, subject to shared usage limits.

Included allowance; optional additional usage billed separately.

Claude and Claude Code share plan limits. Higher effort consumes more allowance; exact quota varies by plan and task. Claude and Claude Code route flagged cybersecurity requests to Opus 4.8 and flagged biology/frontier-LLM requests to Opus 5 by default; benchmark runs that do not disclose the setting are fallback-unknown.

Opus 5.5 announcement ↗ · Claude Code plans ↗ · Automatic model switching ↗

API Verified route

claude-opus-5-5

$4 input / $20 output USD per million tokens.

Adaptive thinking is always on; supported efforts are low, medium (default), high, xhigh and max. AA rows use default safety fallback; standalone or fallback-unknown results are not interchangeable.

API model ↗

Self-hosting Not verified

No verified open-weight or self-hosting route for this exact model.

Not verified

Do not infer exact-version availability or included usage from a family name.

API model ↗

Method sensitivity & winning margins

These are declared scenarios, not confidence percentages. They vary Elo slope by half/double, group weights by 0.5/1/2, omit benchmarks/groups. Your setup and exclusions apply. Priced results apply your spending ceiling; quality-only results ignore costs.

Fixed cohorts keep the baseline eligible configurations; expanded cohorts admit configurations newly scoreable under each composition. Validated estimates stay fixed while weights vary. Native margins below include only measured pairs; these scenarios do not test estimation uncertainty. Several components share Artificial Analysis, model judges and agent tools; they are not independent confirmations. Benchmark-specific harness and fallback limits remain.

Baseline: 109 priced configurations from 33 models. Claude Opus 5.5 max leads Claude Opus 5.5 xhigh by 1.494 points.

Briefcase analytical quality: native margin +49.4100 at 30.0% weight.
AA-LCR 1.1: native margin 0.0000 at 30.0% weight.
AA-Omniscience index: native margin +3.7667 at 40.0% weight.

Evidence for Figure it out

27 models

Relevant benchmark strengths, including setups that cannot set a priced recommendation. Supporting benchmarks stay separate from the profile score.

Claude Fable 5Scientific reasoning · Factual reliabilityComparable profile option · One publisher

At least one qualifying effort has complete, priced profile evidence.

CritPt ↗max effort · 28.57 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-Omniscience index ↗max effort · 43.30 points · $0.070 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18

Claude Fable 5.1Financial analysis · Practical analysis +5Comparable profile option · One publisher

At least one qualifying effort has complete, priced profile evidence.

Financial analysis · Practical analysis · Scientific reasoning · Reading documents · Hard reasoning · Long-context reasoning · Factual reliability

AA-AnalystAgent pass@1 ↗max effort · 65.75 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
Briefcase analytical quality ↗max effort · 1965.92 Elo · $22.71 / benchmark taskQuality–cost frontier · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗max effort · 29.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗xhigh effort · 31.14 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗high effort · 30.29 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗medium effort · 29.14 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗low effort · 27.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
GDP.pdf all-pass ↗low effort · 28.00 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
HLE ↗max effort · 59.13 points · $1.59 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-10-18
HLE ↗xhigh effort · 58.71 points · $1.03 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-10-18
AA-LCR 1.1 ↗max effort · 85.33 points · $1.58 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-LCR 1.1 ↗high effort · 83.67 points · $1.48 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-LCR 1.1 ↗medium effort · 84.67 points · $1.48 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-Omniscience index ↗max effort · 43.45 points · $0.27 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-Omniscience index ↗xhigh effort · 42.38 points · $0.077 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18

Claude Haiku 5.5Financial analysisComparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA now measures low through max as standalone efforts, matching the documented absence of server-side fallback. Lower-tier tariffs are restored, but the exact-row caveat says 5x long-context pricing is omitted. Aggregate tokens do not establish request-tier allocations; native benchmark bills stay missing. Figure uses a separately marked conditional tier envelope and non-expected midpoint; Execute predicts missing DeepSWE separately and has no established total-bill bound. AA flags pre-release AutomationBench refusals pending reevaluation. Direct DeepSWE is unmeasured; fallback-unknown Claude Code and fallback-labelled Zapier results join nothing.

AA-AnalystAgent pass@1 ↗max effort · 68.25 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · single model · 2026-10-18

Claude Opus 5Scientific reasoningComparable profile option · One publisher

At least one qualifying effort has complete, priced profile evidence.

CritPt ↗max effort · 29.14 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · single model · 2026-10-18
CritPt ↗xhigh effort · 27.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · single model · 2026-10-18
CritPt ↗high effort · 28.29 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · single model · 2026-10-18

Claude Opus 5.5Practical analysis · Scientific reasoning +4Comparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA measures low through max with default safety fallback. AA publishes an Opus 4.8 fallback route for max only, without fractions for these benchmarks, so max costs are cost bounds and lower efforts are primary-tariff scenarios. No matching direct DeepSWE result exists.

Practical analysis · Scientific reasoning · Reading documents · Hard reasoning · Long-context reasoning · Factual reliability

Briefcase analytical quality ↗max effort · 2142.71 Elo · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
Briefcase analytical quality ↗xhigh effort · 2093.30 Elo · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗max effort · 31.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗xhigh effort · 31.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗high effort · 30.86 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗medium effort · 27.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
GDP.pdf all-pass ↗high effort · 28.80 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
HLE ↗max effort · 61.35 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
HLE ↗xhigh effort · 57.51 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-LCR 1.1 ↗max effort · 84.67 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-LCR 1.1 ↗xhigh effort · 84.67 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-LCR 1.1 ↗medium effort · 84.33 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-Omniscience index ↗max effort · 46.42 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
AA-Omniscience index ↗xhigh effort · 42.65 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18

Claude Sonnet 5.5Financial analysis · Practical analysis +1Comparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA measures low through max with default safety fallback, which Anthropic routes to Sonnet 5, whose input/output list prices match. AA captures the $0.10 cache-read rate, reduced October 7. AA publishes no fallback route or fractions, so costs are primary-tariff scenarios. FrontierCode Claude Code runs do not disclose fallback use and join nothing. No matching direct DeepSWE result exists.

Financial analysis · Practical analysis · Scientific reasoning

AA-AnalystAgent pass@1 ↗max effort · 69.50 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
Briefcase analytical quality ↗max effort · 2115.09 Elo · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗max effort · 31.43 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18
CritPt ↗xhigh effort · 31.14 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-10-18

How options qualify & what this search misses

Within your available configurations, each benchmark admits results within 5 points of its leader (100 for Elo), plus quality–cost frontier results within 20 points (400 Elo). These are screening choices, not confidence intervals. No cost means no value claim. Known weak results remain available in each source; a strength is not an endorsement of the whole model.

The 2026-10-18 survey examined 704 AA configurations, 136 FrontierCode entries and 126 Zapier entries. Coverage and public availability remain incomplete; this is a source-backed shortlist, not proof that every great model was found.

Access reference for all 44 maintained models

Choose a model

Access: subscription, API or your hardware

Reviewed 2026-09-10. Availability does not reproduce a benchmark’s agent, effort or fallback configuration.

Subscription Verified route

Plus and Business Standard include limited Astra usage; Pro and Business Premium can use their full existing Work/Codex allowance on Astra.

Plan subscription plus optional credits; task/model-dependent allowance.

Work and Codex share usage; Chat has separate allowances. Optional purchased credits extend usage. Inclusion is not unlimited usage or zero API task cost.

Codex plans ↗ · Work and Codex plans ↗

API Verified route

gpt-6-astra

$10 input / $50 output USD per million tokens; standard API billing.

API efforts: low, medium, high, xhigh, max. Harness menus may differ. Long-context surcharges and caching terms are on the model page. Ultra is a separate multiagent system; benchmark scores do not transfer to it.

API model ↗

Self-hosting Not verified

No verified open-weight or self-hosting route for this exact model.

Not verified

Do not infer exact-version availability or included usage from a family name.

API model ↗

Use your allowance wellchat, agent and when to switch

Practical guidance for each kind of work, from the configuration the rail names. There is no measured quota optimum.

Use your allowance well.

Settle the decision
Decide what success means with the model that has the useful context.
Your agentClaude Opus 5.5 · max · Estimated
Start here when the investigation needs files, tests or tools.

Limits & when to switch

On Claude Pro and Max, Chat and Claude Code share usage limits. Switching between them adds no separate allowance. Max and Team include monthly Claude Console API credits, which do not cover Claude Code or app extra usage or change those limits.

Skip a separate chat when the plan is settled or the decision needs repository tools. Reconsider the model or effort when the design is unresolved or fixes keep failing; carry the useful context with you.

Move the rail to inspect the trade-off. API savings do not measure subscription savings, and the benchmarks do not identify when to escalate.

Selected model access: Available in Claude and Claude Code paid plans, subject to shared usage limits. Check access & evidence below.

Product facts checked 2026-10-08. Opus 5.5 model access Claude Pro and Max usage Max and Team API credits

Practical guidance · no measured quota optimum.

Use your allowance well.

Settle the decision
Decide what success means with the model that has the useful context.
Claude CodeClaude Sonnet 5.5 · max · Estimated
Bring the decision. Build, verify and fix in the same task.

Limits & when to switch

Move the rail to inspect the trade-off. API savings do not measure subscription savings, and the benchmarks do not identify when to escalate.

Selected model access: Available in Claude apps and Claude Code, where the default effort is medium. Check access & evidence below.

Product facts checked 2026-10-08. Opus 5.5 model access Claude Pro and Max usage Max and Team API credits

Practical guidance · no measured quota optimum.

Subscription, API or self-hostingwhat the cost numbers mean

Benchmark API cost is a comparison unit, not your bill. Subscription plans may include an allowance, require credits from the first request, restrict supported tools or leave exact versions undisclosed. Max and Team now include monthly Claude Console API credits; these do not cover Claude Code or app extra usage or change their limits. The selected model’s access disclosure and the full access reference keep those distinctions separate, with official sources and verification dates.

OpenAI’s October system card distinguishes October Chat Sol/Luna checkpoints from September versions in Work and Codex. The Chat variants do not inherit the measurements here; official help and launch pages also disagree about Chat availability and rollout scope.

Downloadable weights establish a self-hosting route, not laptop suitability or savings. The reference gives sourced serving examples and distinguishes author formats, community quantizations and related hosted variants. For example, hosted Qwen Flash is based on Flash-Next with additional production features; it is not silently treated as the same measured weights configuration. Local hardware, electricity and operations costs are not estimated here.

What turning the effort dial up costsmeasured steps and regressions

Turning the dial up buys the model more thinking. You pay for that thinking in tokens, so it is a spending dial first and a quality dial second.

That makes it measurable. Every row here is one model at one setting, scored on DeepSWE v1.1 · mini-swe-agent, Terminal-Bench 4.0 · AA harness, GDPval-AA v2.1 and AutomationBench-AA · partial credit and billed with the same job weights. The ladder compares complete measured settings; a missing setting is skipped, not estimated.

GPT-5.6 Terra shows it most starkly. Among models with at least four complete settings, the last step costs about 10× what the first did: $0.020 a point at the bottom, $0.20 at the top. Measured October 18, 2026.

GPT-5.6 Terra — each step of the reasoning dial, on the execution lane

Step	Score	Points bought	Added cost	$ / point
low → medium	23.26	+5.70	$0.1113	$0.0195
medium → high	32.72	+9.46	$0.3694	$0.0390
high → xhigh	39.41	+6.69	$0.7733	$0.1156
xhigh → max	53.93	+14.52	$2.8868	$0.1988

No number here tells you where to stop. The board prices a point at each step. What a wrong answer costs you is the other half, and only you have it.

Higher does not always mean better, either. 2 measured configurations score lower at a higher setting:

GPT-6 Astra · xhigh → max — 0.25 points down, $2.33 more per task. Paying more for a worse result.
Grok 4.6 · high → xhigh — 0.61 points down, $0.76 more per task. Paying more for a worse result.

These measured component scores do not include benchmark error bars. Read that as a reason to check your own setting, not as a law about the model. Either way, “turn it to max” is a guess rather than advice.

Never pick the setting on its own. A cheap model turned up and a costly one turned down are the same kind of thing here — which is why the card compares configurations rather than ranking model names.

What each task is measured onsources, weights and missing evidence

Two owned mixes define the work. Their weights are reasoned policy from a two-week sample of the author's own agent work (213 sessions, 2026-09-03 to 2026-09-16), changed as little as that sample allowed. They are not scientifically established constants. Publisher composite indexes cannot change these formulas or decide which effort settings appear.

Figure it out: 40% AA-Omniscience, 30% Briefcase analytical quality, 30% AA-LCR v1.1.
Execute it: 40% direct DeepSWE v1.1, 35% AA Terminal-Bench 4.0, 15% AutomationBench-AA partial credit, 10% GDPval-AA v2.1.

What scores the Execute lane

Direct DeepSWE v1.1 measures repository work in mini-swe-agent, using pass@1 over scored attempts on its 113-task set. Provider, verifier and network errors are excluded by the publisher; context-window failures and agent timeouts count as failures. Its source publishes mean cost over scored attempts. Its publisher prices Astra's five costs at current API tariffs.

AA's Terminal-Bench 4.0 measures command-line work in AA's own scaffold. GDPval-AA v2.1 compares professional deliverables. AutomationBench-AA contributes the fraction of objectives completed, including partial credit; it is not the fraction of whole workflows completed. These are separate source runs from the AA Coding Agent Index.

Together, repository and terminal work carry 75% of Execute, while app workflows and documents retain 25%. Each component keeps its exact model, effort, benchmark version and harness. The mix is a workload proxy across those settings, not a measurement of one universal agent. Changing the provider filter changes access, never the benchmark harness or its score.

Fixed conversions preserve differences without depending on the current roster. Both mixes use:

Rate p                     → 100 × p
Published Omniscience O     → (O + 100) / 2
Native Briefcase/GDPval E   → (E − 500) / 20

Elo is not clipped: 1000 becomes 25 points, 1500 becomes 50 and 2500 becomes 100; higher or negative values remain possible. Twenty Elo per point is a declared cross-benchmark policy, not an empirically established exchange rate. 1000 Elo is not asserted to mean human-level performance. Measured scores do not change when another model is added or filtered. Estimates are re-fitted when the retained measurement corpus changes; UI filters do not re-fit them.

The October 18 capture uses GDPval-AA v2.1 and Briefcase v1.1. AA re-fits both Elo scales within those versions; all current scores use one capture's results together. Differences from earlier captures are not evidence of model improvement. The weights and conversions above stay fixed.

Costs use the same weights as quality. AA bills use benchmark token counts, cache rates, prices and published fallback fractions divided by task count. When fallback pricing is unavailable, a separately marked estimate uses the primary model tariff; it is not a verified fallback-inclusive bill. Briefcase analytical quality uses the full Briefcase task bill. Direct DeepSWE costs are already per scored attempt. The resulting cost prices the declared benchmark mix, not the reader's next job or subscription quota.

Missing results and estimates

A missing measurement stays missing in the source. The estimator tries each missing component against available components in the exact configuration. Quality uses a straight line or a curve that levels off near the top, whichever predicts held-out models better for that pair of benchmarks; cost uses a straight line in log dollars. A predictor score beyond every trained configuration is read at the highest or lowest trained value. A predicted score never goes past the best or below the worst measured score on its benchmark. It never sits above the same model's measured higher effort, or below its measured lower effort, by more than the largest such inversion measured on that benchmark between reasoning efforts; a non-reasoning setting, which is not simply a lower effort, allows the larger of that and the largest inversion measured between non-reasoning and reasoning settings. A prediction held at any of those edges says so. Each training model receives equal total weight. Validation hides all efforts of a model together, and the predictor must beat a model-balanced benchmark-mean baseline across at least six other models. Predictions never train on other predictions or borrow an adjacent effort as a measurement. Sparse rows can show predictions, but a recommendation still needs at least half its score weight directly measured.

An Estimated label and hollow chart point identify a configuration with estimated quality or cost. The evidence disclosure shows the measured share, missing components and estimate basis. Statistical predictions include a predictor, held-out error and range. That range adds weighted component error allowances, each the model-balanced 95th-percentile held-out error among the third of measured configurations whose predictor scores are nearest, widened for predictors outside the training range. With each model left out in turn, those allowances covered 90% of held-out configurations on average across benchmark pairs: 89% in each board's bottom quarter and 89% in its top quarter. A single allowance for the whole board covered 83% in the bottom quarter. Coverage varies by pair: for DeepSWE predicted from Terminal-Bench 4.0 it is 85% overall, with a top-quarter allowance a third as wide as the whole-board one. It is an empirical missing-input range, not a confidence interval. Small measured-score differences also need not matter in practice.

Known token accounting with captured fallback tariffs but missing fractions supplies cost bounds from the cheaper and dearer rates in each token category. Their midpoint is a display convention, not an expected bill. If AA publishes a per-benchmark cost but no fallback tariff, its captured tokens priced at the primary tariff supply a scenario, with the actual fallback-inclusive bill and range unknown. Where a separate benchmark's cost is missing, a validated transfer may use that source-derived scenario as its predictor; the resulting estimate has no defensible bill range. Cost and quality estimates are separate; either may remain unavailable. The rail uses the displayed central estimates, so an estimated cost can exceed its indicative ceiling.

Fallback-enabled systems never inherit standalone or fallback-unknown measurements. A prediction about a system absent from the training overlap remains an explicit extrapolation. Generic model names do not establish dated releases. If the evidence is too sparse or a predictor fails validation, the row remains visible without a composite.

The weekly refresh replaces estimates automatically. A compatible measured component or completed cost accounting takes precedence on regeneration, removing that component's estimate and its error allowance. New benchmark versions receive compatibility and coverage review before activation; old-version measurements never fill a current-version gap. The owned weights remain fixed.

What this composition costs you

Different harnesses still matter. Direct DeepSWE and AA evaluate different scaffolds, tools and tasks. Owning the mix restores useful component coverage but cannot isolate model quality from those environments or predict performance inside Codex or Claude Code. The evidence table retains the source of each number so that trade-off stays inspectable.

Source quality is uneven. DeepSWE uses original engineering tasks and behavior-based verification, but covers a limited repository and language mix. AA's deliverable and analysis comparisons depend on model judges. AutomationBench's partial-credit objectives are easier to satisfy than complete workflows. FrontierCode, Zapier and HLE remain supporting evidence; they do not gate the mixes.

What the Figure lane's own sources are worth

The Figure mix emphasizes analysis, document reasoning and factual reliability. Its sources have their own limits:

HLE is supporting evidence, outside the mix. It has contested answers. FutureHouse examined 321 text-only chemistry and biology questions and found that 29% ± 3.7 had answers contradicted by peer-reviewed literature (FutureHouse, 23 July 2025). The HLE team's own follow-up on that page put roughly 18% of a reviewed subset as problematic under three-expert review and committed to rolling revisions. The finding covers that subset, not the whole exam.
AA-Omniscience's questions are machine-written. Artificial Analysis states its 6,000 questions were generated by an LLM-based question-generation agent from authoritative academic and industry sources (AA, read 2026-09-08). The sources are human; the questions are not.
AA-LCR v1.1 corrected its own answers. AA's methodology records that v1.1 added a grading system prompt, corrected 16 answer keys and changed the grader, and that scores are not directly comparable with v1.0 (AA methodology, read 2026-09-08).
Briefcase is Artificial Analysis's own private benchmark. Its 91 tasks across four held-out scenarios were written by outside experts from firms including Google, McKinsey and BCG, but AA owns and runs it, only a fifth demonstration scenario is public, and AA's methodology table lists one repeat per task — no averaging over attempts (AA-Briefcase, read 2026-09-08). Its analytical-quality Elo is a preference ranking, which is not a correctness measure.

These are benchmark proxies, not success probabilities. Collaborative problem framing and planning remain unmeasured in Figure; selective escalation and supervision remain unmeasured in Execute. Agent harnesses, tools, task sets and judges differ. Shared AA provenance is not independent corroboration, and small numerical gaps need not matter in real work.

Data and limitscoverage, effort names, raw rows and sources

Which models are included

The comparison uses AA model evidence captured October 18, 2026, and direct DeepSWE v1.1, re-read that day; DeepSWE's source payload is unchanged since its September 22 generation. AA also prints a Daybreak Blue Sol max row with cybersecurity results only; it supplies no component of either mix and does not inherit the standard Sol measurements. The table includes the exact effort settings those sources publish for catalogued models, including rows that cannot form a full mix. It excludes rows AA estimates rather than measures, and the settings named below. The shortlist also preserves independent benchmark observations. Every result keeps its source effort, harness and system; measurements do not transfer across them.

An older model stays while something here depends on it: a frontier step, a route, a qualifying shortlist strength or value option, or a substitute for a named gap. Replacement within its vendor line starts that check; age alone never removes it. An open prediction, a recorded estimate that a future measurement would settle, does not keep an older model, since it would settle only if the source measured that model again. On that rule Opus 4.8 and Sonnet 5 left on September 29, replaced by Opus 5.5 and Sonnet 5.5; their open predictions closed unsettled.

Astra has no non-reasoning row in AA's capture, and current OpenAI documentation does not support that configuration. Muse Spark 1.3 max is publicly available and included; missing costs remain visible. Meta now documents native xhigh and Standard-tier Muse Spark 1.3 max; Contributor variants and Muse Code multi-agent runs do not inherit these measurements. The first-party DeepSeek Flash and Flash Vision aliases now serve V4.1 Flash, whose results cannot be inferred from the measured older versions; on October 17 DeepSeek upgraded V4.1 Flash to V4.1-Flash-1017, which AA has not measured, so the V4.1 Flash rows describe the September release rather than what the alias returns today. V4.1 Flash has its own measured max and thinking-disabled rows. GLM-5.3 is measured at low and max; its high setting is a named gap. AA prints partial results for Agnes 2.5 Pro Beta, with an estimated overall index and no Briefcase, AutomationBench or Terminal-Bench 4.0 result, and none for Agnes 3.0 Flash, which is not compared. Agnes 2.5 Pro Beta is not compared either: its AA-LCR, 83.0 at $0.0115 a task, is beaten by GPT-6 Luna max at 83.3 and $0.0102, and its one shortlist observation is on a benchmark neither mix reads. Every measured non-reasoning row of a catalogued model is included like any other effort, including GPT-5.6 Luna, Sol and Terra, GPT-5.5, Qwen3.8 27B and DeepSeek V4 Pro 0813. SWE-2 has scoped FrontierCode evidence without a complete lane mix. Step 5 Preview and Qwen3.8 Max 0902 retain undisclosed AA effort settings: their scoped results do not qualify them for a broad recommendation. Step’s scoped task costs use AA’s captured $0.10 per million cached-input tokens; StepFun’s pricing page now lists the same rate; its October 8 version listed $0.05. Grok 4.7 low, high and xhigh have AA task costs alongside their measured Figure scores; xAI's reasoning guide also lists medium, which is unmeasured. MiMo-V2.6-Pro and MiMo-V2.6-Flash each have measured Figure evidence with thinking enabled, their native default. Grok 4.7 and both MiMo models lack a matching direct DeepSWE result; Execute uses separately labelled estimates where validation permits. Pro and Flash results do not transfer to each other, and UltraSpeed, thinking-disabled use and MiMo's open RL weights are unmeasured.

GPT-6 Sol and Luna each have six exact AA settings, with non-reasoning kept distinct from the five reasoning efforts. Neither has a matching direct DeepSWE result; the Execute lane labels any validated prediction separately. Opus 5.5 has five AA settings with its default safety fallback, and its Figure scores are measured. AA publishes an Opus 4.8 fallback route and tariff for max only, without per-benchmark fractions, so max costs are cost bounds shown at their midpoint. The other efforts' costs are marked primary-tariff scenarios from AA's captured benchmark tokens; they are neither actual bills nor bounded estimates. Opus 5.5 has no matching direct DeepSWE result: its Execute score predicts the missing 40% and its direct task cost is inferred from AA accounting. Its Execute costs carry no range, because the direct DeepSWE share is itself a scenario; the actual bill is unknown.

GPT-6.1 Sol has five exact AA settings, low to max, released September 29. OpenAI's model page lists those five efforts and excludes none and minimal; it mentions no fallback routing, so the rows are treated as standalone. It prices at $2 input, $0.10 cached input, $2.50 cache write and $10 output per million tokens. Its Figure scores are fully measured: max scores 72.3 at $0.76 a task, against Astra max at 71.7 and $3.19, which makes it the Codex Figure pick with no budget limit. The margin is 0.56 points, between two measured scores, so no estimate is involved; 4 of the 35 alternative weightings in the method sensitivity table put Astra max ahead: two that halve reasoning and double reliability while weighting analysis at its usual share or halved, and two that drop AA-LCR. It has no matching direct DeepSWE result, and figures from other sources do not substitute for one, so its Execute score predicts the missing 40%: at max it reads about 63.7 (range 61.0 to 66.5) at about $1.48, under the measured Astra xhigh at 65.9. GPT-6 Sol and GPT-5.6 Sol are superseded by it but stay, since a scoped shortlist strength or value option depends on each.

Grok 4.7 low scores 68.5 on Figure at $0.82 a task, against 70.7 at $2.31 for high. DeepSeek V4 Pro 0813's non-reasoning row, none in the chart, scores 37.4 at $0.15. AA now prices both none and max at DeepSeek's peak rates, $1.32 input and $3.96 output per million tokens. Off-peak rates are half those prices; the chart retains the captured peak basis.

Claude Haiku 5.5, released October 7, has five measured AA efforts, low to max; Anthropic's effort guide makes medium the API default. On October 8 AA removed Default Fallback from the same five row IDs, matching Anthropic's fallback guide: Haiku has no server-side fallback. Its Figure score at max is 68.0. AA restored lower-tier tariffs, but its row-specific caveat says the fivefold long-context pricing is not yet reflected. A separate release note says costs reflect tiered pricing; the captured payload still has only aggregate tokens and lower-tier tariffs, with no request-tier allocations.

Native bills stay missing, but the comparison includes a marked tier scenario. For Haiku max, the standard Figure mix gives $0.21–$1.05 per task, conditional on AA's captured tokens, cache-hit rate, five-minute cache writes and task counts. The rail uses the $0.63 midpoint scenario, not an expected price or an inferred premium share. The comparison shows whether the budget choice changes at the lower endpoint, midpoint and upper endpoint; these readings do not cover every possible request allocation. Other billing modifiers are outside the envelope. Anthropic's model page lists $0.10 input, $0.01 cached input and $0.50 output per million tokens through 100,000 prompt tokens, with fivefold rates above that threshold. AA also flags elevated pre-release AutomationBench refusals and plans reevaluation; the comparison retains the published scores. No effort has a matching direct DeepSWE measurement. Execute predicts that missing component using the existing model-held-out estimator, trained only on other models' measured results. Its cost is seeded from the marked AA tier scenario; the source accounting and transfer errors are not jointly bounded, so the total Execute bill has no established bound.

Mistral Large 4 Preview is a public API preview, released October 6. AA does not disclose its effort, so it remains a scoped alternative at unspecified and cannot become a broad recommendation. Its captured rates are $1.36 input, $0.14 cached input and $4.18 output per million tokens. Mistral advertises a two-week 50% launch discount; the chart keeps AA's base-price evidence. The documented high setting and forthcoming weights do not inherit the unspecified measurement.

Gemini 4 Argon reached general availability in the Gemini API on October 13 (changelog); Google announced it on September 30 for vetted cyber defenders only, and the model id is gemini-4-argon. Google's thinking guide lists low, medium and high, with medium the default. AA measures High only, on a row dated September 30, before general availability, and does not say how it ran the model, so the comparison assumes the row is the model Google now serves. Argon is compared at high only. medium, the default, is a named gap; FrontierCode and Zapier also print medium, supporting evidence that scores neither mix. Costs keep AA's captured rates of $2 input, $0.10 cached input and $10 output per million tokens. Google's pricing page lists those rates through November 30 and doubles all three from December 1, so every Argon cost here doubles on that date. Figure is fully measured at 68.3 and $1.82 a task. Direct DeepSWE has no Argon row, so Execute predicts the missing 40% and reads about 66.2 (range 63.4 to 68.9) at about $4.13. That is a higher estimated score than Astra xhigh's measured 65.9, at an estimated cost (range $3.43 to $5.46) below Astra's measured $4.34, and both ranges include Astra's figures; Argon replaces Astra xhigh as a rung on the All models Execute frontier, while Astra xhigh stays the Codex Execute pick. With Anthropic models excluded, Argon high leads All models Execute on its estimate alone: Astra xhigh leads when every missing DeepSWE result is set to the measured average, and on measured components alone. Against Argon's estimate, Sonnet 5.5 max stays ahead only while its DeepSWE result is above 66.1. Google's own DeepSWE v1.1 figure of 77.9% is a vendor statement and is not scored; the direct board lists no Argon result.

Meta released Muse Wonder on October 12 as muse-wonder-1.0. Meta's model list calls it a preview in the Model API, at $4 input, $0.20 cached input and $20 output per million tokens. The reasoning page lists minimal through max, rejects none, and says max on this model does not select Muse Wonder Full. AA measures max only, so high is a named gap. Figure is fully measured at 71.2 and $3.89 a task, behind Sol 6.1 max at 72.3 and $0.76; Execute predicts the missing DeepSWE share at about 63.6 (range 60.8 to 66.3) at about $9.47. Neither lane puts it on a frontier. Muse Wonder Full, which AA also prints at Max, is for Muse Code Power and Maximum subscribers only, with no API and so no comparable task price. It is a named absence, not a row: its AA results do not transfer to Muse Wonder, and Meta's own DeepSWE claims for the Muse Wonder line, which cover the standard model as well, are not scored.

Ling 3.0 Flash VL was a scoped document-reasoning value option while AA priced it. Its author card publishes separate MIT weights; the vendor changelog dates public API access to September 4, while AA dates the row September 10. AA leaves effort undisclosed, so its measurement does not establish either native thinking setting. AA withdrew its price fields on October 14, so its native task costs are unavailable; the official price page is not substituted, and the measured scores stay without a cost. It has no matching direct DeepSWE result.

Ling 3.1 Flash, JT-4.1 Flash 236B A21B and Solar Mini 4 have AA rows but no result that qualifies for the shortlist, so none is catalogued. Ling 3.0 Flash Fin, a finance-tuned variant whose model card asks for further validation on long workflows, has one value observation, AA-LCR 73.7 at $0.0087 a task; it is deferred: GPT-6 Luna high scores 79.3 on the same benchmark, 5.7 points more, for $0.001 more a task.

Sparse predictions appear for other models, but rows with less than half the score weight measured cannot become recommendations. The two GPT Pro configurations have no exact component evidence to seed model-specific estimates. The shortlist states evidence limits beside each model. The discovery record retains availability checks, omissions and source evidence.

The Execute leader and its limits

With no budget limit, as of the October 18 capture, Sonnet 5.5 max leads Execute for All models and Claude Code. Its score range overlaps the range for Opus 5.5 max, so the lead is not settled, and the answer says close call. On the measured 60% of the score alone it still leads: 63.6% against 59.6% on Terminal-Bench 4.0 and 71.8% against 69.5% on AutomationBench, while GDPval-AA, at 1838 against 1866 Elo, favors Opus 5.5. Neither has a matching direct DeepSWE result, and both are predicted 72.6 on it, just under the best measured 74.1. Opus 5.5 max ties the highest trained Terminal-Bench 4.0 score, 59.6; Sonnet 5.5's 63.6 is above every trained configuration, so it is read at 59.6 too, and its prediction is flagged. Like every fallback-enabled system, each prediction is also an extrapolation from systems measured without a fallback.

For All models the lead also rests on that prediction. The fully measured Astra xhigh scores 65.9; Sonnet 5.5 max stays ahead of it only while its DeepSWE result is above 65.3. Of the configurations with a measured DeepSWE result, the five with the highest Terminal-Bench 4.0 scores, Astra at medium to max and Opus 5 max, reach 72.8 to 74.1, so the lead is plausible but not measured. estimator_sensitivity.py scores the leaders four ways: as published, with a plain straight line and no bounds, with every missing DeepSWE result set to the measured average of 56.7, and on measured components alone. Only the measured average changes the All models leader, to Astra xhigh; the Claude Code leader is the same in all four. For Codex, Astra xhigh leads Execute with a margin of 0.2 points over Astra max. The range for GPT-6.1 Sol max includes Astra xhigh's score, and the plain straight line, which has no bounds, would put GPT-6.1 Sol max first at 67.2, so whether Astra xhigh stays ahead depends on the estimate for GPT-6.1 Sol's missing DeepSWE result.

Sonnet 5.5 has five AA settings with the default safety fallback. Anthropic's Sonnet 5.5 release notes say the fallback sends requests Sonnet 5.5 refuses on cybersecurity or AI-development grounds to Sonnet 5, whose input and output list prices match. Anthropic's platform release notes date Sonnet 5.5's cache-read reduction from $0.20 to $0.10 per million tokens to October 7; AA's capture uses the new rate. AA publishes no route or token split for those retries, so how many tokens the retries use, and whether AA's token counts include them, are unknown; every Sonnet 5.5 AA cost is therefore a primary-tariff scenario without a range. At max, Sonnet 5.5 comes to $7.37 per Execute task and Opus 5.5 to $10.51. Both include an estimated cost for the missing DeepSWE share, predicted from each model's own Terminal-Bench 4.0 cost. Each price therefore follows its own model's AA basis: the primary tariff for Sonnet 5.5, and the midpoint of the cost bounds for Opus 5.5 max. The gap between them says little, and neither is a measured Execute bill.

Supporting benchmarks and missing results

Additional evidence can qualify a scoped shortlist alternative without changing the broad score. Figure includes GDP.pdf, CritPt, AnalystAgent and ITBench. Execute includes FrontierCode, Zapier, SciCode, EnterpriseOps-Gym and APEX-Agents. Supporting observations never replace a missing core component. Coverage varies across exact configurations; not measured is a missing result, not poor performance.

AA-Briefcase separately reports analytical quality and presentation. A low presentation score does not arithmetically lower its separate analytical-quality Elo. Its rubric, analytical Elo, and overall Elo answer different questions; none is silently substituted for another here.

Where the numbers come from

Artificial Analysis, direct DeepSWE, Cognition, and Zapier supply the measurements and accounting. The retained source captures and reproduction commands show how those inputs become the two lane scores, direct costs, and recommendations; the direct DeepSWE capture retains the source JSON, its hash and its scoring and cost limits. Every page is parsed as data — no downloaded script is executed. Unknown source versions stop regeneration for review. Missing measurements or task accounting remain visible; a track can be selected only when its mix is measured or completed by a validated estimate.

Historical calibrated predictions remain in the dated research and falsifiability records. They do not enter the current comparison or choose a recommendation.

Effort dials, model by model

GPT-6 Astra has measured low, medium, high, xhigh, and max configurations here. OpenAI's model guide excludes none; Pro is a separate class.
GPT-6.1 Sol has measured low, medium, high, xhigh, and max configurations here, with medium the API default. OpenAI's model guide excludes none and minimal.
Claude's effort documentation distinguishes model-specific effort settings. The measured Fable configurations include the source's fallback behavior; the direct bill accounts for those fractions.
Gemini Flash uses low, medium, and high thinking levels here. Each is independently measured. Gemini 4 Argon uses the same three levels, with medium the default; only high is measured, and medium is a named gap.
Muse Wonder documents minimal through max and rejects none; only max is measured, and high is a named gap.
DeepSeek V4.1 Flash exposes public API low, high (default), and max. medium and xhigh map to high; ultra maps to max; none disables thinking. Max and thinking disabled are measured here, on the September release; the alias has served V4.1-Flash-1017 since October 17.
Earlier DeepSeek versions and GLM document low / high / max; an unmeasured setting stays a named gap.
Qwen3.8-Flash-Next distinguishes its effort dial from disabling thinking. Only the measured xhigh configuration is placed here.

What the prices do not include

Benchmark API cost is not your invoice. It includes the evaluated runs' recorded API usage, but does not price your extra retries, human review, latency, subscriptions, or the cost of a wrong answer. An API cost ratio is not a subscription-quota ratio. The rail prices a benchmark task mix as shares of a job; it does not forecast the cost of your next job, and the job split it uses is a control you set, not a measurement of you.

121 catalogued configurations · October 18, 2026 · ordered by Figure score. Each lane uses its declared job weights. ≈ marks an estimated score or cost; row details give its basis, any available range, measured gaps and recommendation eligibility. These scores are benchmark proxies, not probabilities of success.

Model · effort	Figure	$ / figure task	Execute	$ / execute task
Claude Opus 5.5 · max Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 79.3; no score component estimated. API cost ≈$9.27 per mix task; range $6.58–$11.96. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $29.90 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.76 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.18 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈67.2; missing-input range 64.4–69.9 points. API cost ≈$10.51 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $6.51 per scored attempt, estimated. Predicted from the named predictor benchmark’s fallback allocation midpoint, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $18.40 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $12.14 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.71 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	79.32	≈$9.2711	≈67.16	≈$10.5141
Claude Opus 5.5 · xhigh Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 77.8; no score component estimated. API cost ≈$3.86 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $12.27 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.60 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.013 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈66.3; missing-input range 63.5–69.0 points. API cost ≈$5.34 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $4.29 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $8.78 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $4.21 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.88 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	77.83	≈$3.8649	≈66.29	≈$5.3399
Claude Fable 5.1 · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Execute estimate 60% of score weight measured. Score ≈62.2; missing-input range 59.4–64.9 points. API cost ≈$10.86 per mix task; range $6.80–$17.34. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $8.12 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 14 training models, 43 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.74. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $18.07 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $2.05 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	76.28	$7.3967	≈62.18	≈$10.8558
Claude Sonnet 5.5 · max Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 75.5; no score component estimated. API cost ≈$6.30 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $20.47 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.34 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.15 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈68.8; missing-input range 65.9–71.6 points. API cost ≈$7.37 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $5.27 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $12.65 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $6.84 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.03 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	75.49	≈$6.3026	≈68.76	≈$7.3714
Claude Opus 5.5 · high Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 75.2; no score component estimated. API cost ≈$2.06 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $6.27 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.59 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0085 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈64.2; missing-input range 61.5–67.0 points. API cost ≈$3.32 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.16 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $5.12 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $1.54 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.70 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	75.17	≈$2.0622	≈64.25	≈$3.3156
Claude Fable 5.1 · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Execute estimate 60% of score weight measured. Score ≈63.0; missing-input range 60.2–65.7 points. API cost ≈$9.39 per mix task; range $6.03–$14.18. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $8.12 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 14 training models, 43 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.74. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $14.74 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.74 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	74.79	$5.8215	≈62.99	≈$9.3873
Claude Opus 5.5 · medium Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 74.0; no score component estimated. API cost ≈$1.50 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $4.40 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.59 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0075 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈61.8; missing-input range 59.0–64.5 points. API cost ≈$2.70 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $2.77 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $4.04 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.86 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.64 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	73.99	≈$1.4995	≈61.79	≈$2.7037
Claude Fable 5.1 · high Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Execute estimate 60% of score weight measured. Score ≈61.0; missing-input range 58.2–63.7 points. API cost ≈$6.54 per mix task; range $4.21–$9.71. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $5.39 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 14 training models, 43 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.74. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $10.95 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.40 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	73.31	$3.8722	≈60.97	≈$6.5438
Claude Opus 5 · max	72.43	$5.5767	61.20	$12.3602
GPT-6.1 Sol · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈63.7; missing-input range 61.0–66.5 points. API cost ≈$1.48 per mix task; range $1.02–$2.75. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.77 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	72.30	$0.7603	≈63.75	≈$1.4784
Claude Opus 5 · xhigh	72.27	$4.5100	59.46	$10.3722
Claude Fable 5.1 · medium Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Execute estimate 60% of score weight measured. Score ≈57.7; missing-input range 54.9–60.4 points. API cost ≈$5.07 per mix task; range $3.26–$7.48. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 71.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $4.04 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 14 training models, 43 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.74. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $8.68 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.30 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	71.98	$3.0227	≈57.67	≈$5.0675
GPT-6 Astra · max	71.74	$3.1886	65.61	$6.6658
Claude Sonnet 5.5 · xhigh Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 71.5; no score component estimated. API cost ≈$2.19 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $6.98 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.30 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0085 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈64.9; missing-input range 62.2–67.7 points. API cost ≈$3.78 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.48 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $6.08 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $1.95 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.42 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	71.50	≈$2.1851	≈64.91	≈$3.7799
Muse Wonder · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈63.6; missing-input range 60.8–66.3 points. API cost ≈$9.47 per mix task; range $8.14–$12.24. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $6.42 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	71.23	$3.8863	≈63.57	≈$9.4701
Grok 4.7 · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈52.1; missing-input range 48.7–55.5 points. API cost ≈$8.20 per mix task; range $7.01–$10.67. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 67.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $5.71 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	71.08	$2.8783	≈52.08	≈$8.2007
GPT-6 Astra · xhigh	70.85	$2.2801	65.86	$4.3358
Claude Fable 5 · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute estimate 60% of score weight measured. Score ≈56.9; missing-input range 55.6–59.6 points. API cost ≈$13.60 per mix task; range $8.92–$19.92. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 71.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $8.12 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 14 training models, 43 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.74. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $26.18 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	70.81	$7.1887	≈56.90	≈$13.6038
Grok 4.7 · high Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈51.3; missing-input range 47.9–54.7 points. API cost ≈$5.53 per mix task; range $4.71–$7.09. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 67.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $4.33 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	70.69	$2.3058	≈51.28	≈$5.5302
GPT-6 Astra · high	70.39	$1.7279	63.31	$3.4234
GPT-6.1 Sol · high Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈61.5; missing-input range 58.8–64.3 points. API cost ≈$0.82 per mix task; range $0.50–$1.86. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 71.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.13 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	70.24	$0.3103	≈61.54	≈$0.8193
Claude Opus 5 · high	70.12	$3.3502	58.73	$7.4928
GPT-6.1 Sol · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈63.0; missing-input range 60.2–65.7 points. API cost ≈$0.96 per mix task; range $0.61–$2.14. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.28 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	70.06	$0.3729	≈62.95	≈$0.9635
Claude Fable 5.1 · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Execute estimate 60% of score weight measured. Score ≈55.1; missing-input range 52.4–57.9 points. API cost ≈$4.01 per mix task; range $2.62–$5.78. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 70.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.11 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 14 training models, 43 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.74. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $6.99 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.20 per benchmark task, estimated. Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	69.71	$2.4605	≈55.11	≈$4.0128
GPT-6 Astra · medium	69.55	$1.4740	61.06	$3.1395
GPT-6.1 Sol · medium Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈59.6; missing-input range 56.8–62.3 points. API cost ≈$0.65 per mix task; range $0.38–$1.53. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 71.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.96 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	68.59	$0.1973	≈59.57	≈$0.6498
Grok 4.7 · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈45.3; missing-input range 41.9–48.7 points. API cost ≈$3.86 per mix task; range $3.18–$5.14. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 64.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.57 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	68.46	$0.8204	≈45.31	≈$3.8573
Claude Sonnet 5.5 · high Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 68.3; no score component estimated. API cost ≈$1.07 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $3.27 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.29 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0054 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈58.0; missing-input range 55.3–60.7 points. API cost ≈$1.66 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 71.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.96 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $2.19 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.69 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.31 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	68.30	≈$1.0715	≈58.00	≈$1.6638
Qwen3.8 Max 0902 · unspecified Evidence and eligibility Figure excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈56.1; missing-input range 53.3–58.8 points. API cost ≈$9.67 per mix task; range $8.31–$12.51. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 70.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $6.56 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	68.29	$5.9453	≈56.07	≈$9.6747
Gemini 4 Argon · high Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈66.2; missing-input range 63.4–68.9 points. API cost ≈$4.13 per mix task; range $3.43–$5.46. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 72.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.71 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	68.28	$1.8231	≈66.18	≈$4.1284
Claude Haiku 5.5 · max Evidence and eligibility Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · Omniscience reliability: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · AutomationBench-AA · partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure estimate 100% of score weight measured. Score 68.0; no score component estimated. API cost ≈$0.63 per mix task; conditional tier envelope $0.21–$1.05. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $2.02 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.058 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.019 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈50.2; missing-input range 46.8–53.6 points. API cost ≈$1.51 per mix task; range unknown, because part of the cost is an accounting scenario. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.84 per scored attempt, estimated. Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $1.96 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.67 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.13 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.	67.98	≈$0.6323midpoint scenario · conditional $0.2108–$1.0538	≈50.20	≈$1.5054scenario + prediction · bill range unknown
Kimi K3 · max	67.95	$2.1661	45.73	$3.9523
Muse Spark 1.3 · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈54.1; missing-input range 50.7–57.5 points. API cost ≈$4.19 per mix task; range $3.48–$5.54. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.75 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	67.89	$1.2667	≈54.09	≈$4.1886
Claude Opus 5 · medium	67.84	$1.7986	52.69	$4.8090
Grok 4.6 · xhigh	67.64	$1.5282	48.48	$4.8498
Grok 4.6 · high	67.58	$1.2518	49.09	$4.0936
GPT-6 Sol · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈58.1; missing-input range 55.4–60.9 points. API cost ≈$2.68 per mix task; range $2.16–$3.68. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 71.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $2.75 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	67.30	$0.8575	≈58.13	≈$2.6837
MiMo-V2.6-Pro · reasoning Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈54.9; missing-input range 52.1–57.6 points. API cost ≈$0.49 per mix task; range $0.27–$1.22. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.79 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	67.05	$0.1299	≈54.86	≈$0.4920
Grok 4.6 · medium	66.09	$0.8887	46.62	$3.3272
Claude Opus 5.5 · low Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 66.1; no score component estimated. API cost ≈$0.52 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $1.15 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.59 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0058 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈50.3; missing-input range 46.9–53.7 points. API cost ≈$1.58 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.90 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $2.08 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.21 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.49 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	66.08	≈$0.5241	≈50.26	≈$1.5831
GLM-5.3 · max	65.79	$1.7438	57.34	$4.7146
GPT-6 Astra · low	65.66	$0.7207	54.80	$1.6604
Muse Spark 1.3 · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈45.8; missing-input range 42.4–49.2 points. API cost ≈$3.79 per mix task; range $3.12–$5.06. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 64.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.53 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	65.60	$1.0719	≈45.81	≈$3.7894
Step 5 Preview · unspecified Evidence and eligibility Figure excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈52.6; missing-input range 49.2–56.0 points. API cost ≈$3.22 per mix task; range $2.75–$3.96. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.18 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	65.41	$0.7480	≈52.59	≈$3.2158
GPT-5.6 Sol · max	65.17	$1.3607	57.59	$6.6141
Claude Haiku 5.5 · xhigh Evidence and eligibility Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · Omniscience reliability: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · AutomationBench-AA · partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure estimate 100% of score weight measured. Score 64.4; no score component estimated. API cost ≈$0.34 per mix task; conditional tier envelope $0.11–$0.57. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $1.10 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.046 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0024 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈48.2; missing-input range 44.8–51.6 points. API cost ≈$1.13 per mix task; range unknown, because part of the cost is an accounting scenario. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 68.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.51 per scored attempt, estimated. Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $1.38 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.28 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.083 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.	64.35	≈$0.3440midpoint scenario · conditional $0.1147–$0.5733	≈48.23	≈$1.1275scenario + prediction · bill range unknown
GPT-6 Sol · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈52.2; missing-input range 48.8–55.6 points. API cost ≈$1.51 per mix task; range $1.04–$2.81. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.81 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	64.23	$0.4020	≈52.24	≈$1.5093
Claude Sonnet 5.5 · medium Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 64.2; no score component estimated. API cost ≈$0.51 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $1.39 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.29 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0029 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈50.3; missing-input range 46.9–53.8 points. API cost ≈$1.27 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 68.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.63 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $1.58 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.32 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.26 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	64.18	≈$0.5063	≈50.34	≈$1.2749
GPT-6.1 Sol · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈50.4; missing-input range 47.0–53.8 points. API cost ≈$0.46 per mix task; range $0.26–$1.14. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 69.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.73 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	63.86	$0.1107	≈50.39	≈$0.4615
GPT-5.6 Sol · xhigh	63.46	$1.0537	50.59	$3.3859
GPT-6 Sol · high Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈49.9; missing-input range 46.5–53.3 points. API cost ≈$1.30 per mix task; range $0.88–$2.48. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 68.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.64 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	63.37	$0.2517	≈49.91	≈$1.2998
Qwen3.8-Flash-Next · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈50.0; missing-input range 46.6–53.4 points. API cost ≈$0.81 per mix task; range $0.48–$1.89. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 67.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.17 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	62.61	$0.4875	≈49.98	≈$0.8090
GLM-5.3-Flash · max	62.43	$0.3005	51.61	$0.5424
Grok 4.6 · low	62.37	$0.2658	30.29	$1.1439
Claude Haiku 5.5 · high Evidence and eligibility Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · Omniscience reliability: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · AutomationBench-AA · partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure estimate 100% of score weight measured. Score 62.3; no score component estimated. API cost ≈$0.21 per mix task; conditional tier envelope $0.072–$0.36. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $0.67 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.045 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0013 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈43.9; missing-input range 40.5–47.3 points. API cost ≈$0.87 per mix task; range unknown, because part of the cost is an accounting scenario. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 66.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.26 per scored attempt, estimated. Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $0.99 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.13 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.068 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.	62.33	≈$0.2146midpoint scenario · conditional $0.0715–$0.3577	≈43.92	≈$0.8734scenario + prediction · bill range unknown
Claude Opus 5 · low	62.24	$0.7551	44.22	$2.6600
Muse Spark 1.2 · xhigh	62.04	$0.6134	35.38	$3.3875
GPT-5.6 Sol · high	61.68	$0.7420	48.32	$2.3884
DeepSeek V4.1 Flash · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈52.5; missing-input range 49.1–55.9 points. API cost ≈$1.01 per mix task; range $0.66–$1.99. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 68.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.37 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	60.96	$0.1834	≈52.48	≈$1.0072
Claude Sonnet 5.5 · low Evidence and eligibility Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark Figure · Omniscience reliability: Fallback token fractions are not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark Execute · AutomationBench-AA · partial credit: Fallback token fractions are not published for this benchmark Figure estimate 100% of score weight measured. Score 60.8; no score component estimated. API cost ≈$0.33 per mix task; fallback-inclusive bill and range unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $0.80 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.29 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0027 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈44.6; missing-input range 41.2–48.0 points. API cost ≈$1.09 per mix task; range unknown, because part of the cost is an accounting scenario. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 66.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.45 per scored attempt, estimated. Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here. Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $1.28 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.21 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.24 per benchmark task, estimated. Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown. Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	60.79	≈$0.3293	≈44.56	≈$1.0867
Claude Haiku 5.5 · medium Evidence and eligibility Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · Omniscience reliability: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · AutomationBench-AA · partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure estimate 100% of score weight measured. Score 60.5; no score component estimated. API cost ≈$0.13 per mix task; conditional tier envelope $0.044–$0.22. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $0.40 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.045 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.00099 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈38.9; missing-input range 35.5–42.3 points. API cost ≈$0.59 per mix task; range unknown, because part of the cost is an accounting scenario. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 63.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.93 per scored attempt, estimated. Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $0.58 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.062 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.057 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.	60.55	≈$0.1335midpoint scenario · conditional $0.0445–$0.2224	≈38.94	≈$0.5892scenario + prediction · bill range unknown
GPT-6 Sol · medium Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈45.7; missing-input range 42.3–49.1 points. API cost ≈$0.98 per mix task; range $0.61–$2.22. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 65.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.34 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	60.34	$0.1576	≈45.66	≈$0.9847
Gemini 3.8 Flash · medium	60.04	$0.6253	49.04	$2.5781
Gemini 3.8 Flash · high	59.88	$0.8455	50.10	$3.2226
GPT-5.5 · xhigh	59.60	$1.7607	43.29	$7.5060
GPT-5.6 Sol · medium	58.88	$0.4095	41.86	$1.4701
Qwen3.8 27B · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈34.8; missing-input range 24.7–45.0 points. API cost ≈$2.62 per mix task; range $2.10–$3.60. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 52.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $2.73 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	58.59	$0.8718	≈34.84	≈$2.6205
Mistral Large 4 Preview · unspecified Evidence and eligibility Figure excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈50.2; missing-input range 46.8–53.6 points. API cost ≈$3.53 per mix task; range $3.03–$4.32. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 68.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.40 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	58.49	$0.7620	≈50.24	≈$3.5264
GPT-5.5 · high	58.39	$1.1771	39.72	$4.4077
GPT-6 Luna · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈41.8; missing-input range 32.6–51.1 points. API cost ≈$0.32 per mix task; range $0.16–$0.84. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 62.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.56 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	58.25	$0.0551	≈41.84	≈$0.3203
GPT-5.6 Terra · max	57.56	$0.9404	53.93	$4.4476
DeepSeek V4 Pro 0813 · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈43.4; missing-input range 40.0–46.8 points. API cost ≈$2.49 per mix task; range $1.98–$3.44. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 63.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $2.66 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	57.51	$0.3453	≈43.43	≈$2.4856
Gemini 3.7 Flash · high	57.40	$0.6748	44.64	$2.5475
Gemini 3.7 Flash · medium Evidence and eligibility Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark Figure estimate 70% of score weight measured. Score ≈57.4; missing-input range 49.6–57.4 points. API cost ≈$0.37 per mix task; range $0.069–$2.88. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 25.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 107 configurations. Held-out mean absolute error 13.02 points; benchmark-mean baseline 15.33. A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.7 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $1.16 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 90 configurations. Held-out mean absolute error 0.98 log dollars; benchmark-mean baseline 1.32. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 50% of score weight measured. Score ≈44.3; missing-input range 37.1–46.2 points. API cost ≈$2.01 per mix task; range $1.32–$3.84. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 15.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 36 training models, 105 configurations. Held-out mean absolute error 10.23 points; benchmark-mean baseline 15.51. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $3.04 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 30 training models, 83 configurations. Held-out mean absolute error 0.43 log dollars; benchmark-mean baseline 1.30. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 55.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 5.92 points; benchmark-mean baseline 10.39. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.38 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.90. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	≈57.36	≈$0.3743	≈44.29	≈$2.0078
DeepSeek V4 Flash Vision · max Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈41.2; missing-input range 32.0–50.5 points. API cost ≈$1.08 per mix task; range $0.70–$2.12. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 61.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.45 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	57.31	$0.2580	≈41.22	≈$1.0767
MiMo-V2.6-Flash · reasoning Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈49.9; missing-input range 46.5–53.3 points. API cost ≈$0.31 per mix task; range $0.16–$0.82. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 67.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.55 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	57.17	$0.0514	≈49.91	≈$0.3103
GPT-5.5 · medium	56.02	$0.7398	33.03	$2.4265
GPT-5.6 Luna · max	55.83	$0.1292	43.27	$1.5332
Gemini 3.7 Flash · low Evidence and eligibility Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark Figure estimate 70% of score weight measured. Score ≈55.7; missing-input range 47.1–55.7 points. API cost ≈$0.37 per mix task; range $0.067–$2.82. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 25.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 107 configurations. Held-out mean absolute error 13.02 points; benchmark-mean baseline 15.33. A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.7 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $1.14 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 90 configurations. Held-out mean absolute error 0.98 log dollars; benchmark-mean baseline 1.32. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 50% of score weight measured. Score ≈37.6; missing-input range 30.8–41.1 points. API cost ≈$1.38 per mix task; range $0.91–$3.45. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 13.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 36 training models, 105 configurations. Held-out mean absolute error 10.23 points; benchmark-mean baseline 15.51. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $1.58 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 30 training models, 83 configurations. Held-out mean absolute error 0.43 log dollars; benchmark-mean baseline 1.30. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 48.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 5.92 points; benchmark-mean baseline 10.39. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.34 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.90. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	≈55.74	≈$0.3661	≈37.58	≈$1.3752
GPT-5.6 Terra · xhigh	55.64	$0.6091	39.41	$1.5608
GPT-6 Luna · xhigh Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈37.2; missing-input range 28.0–46.4 points. API cost ≈$0.28 per mix task; range $0.14–$0.75. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 57.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.51 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	55.06	$0.0281	≈37.20	≈$0.2796
Gemini 3.8 Flash · low Evidence and eligibility Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Figure estimate 100% of score weight measured. Score 55.1; no score component estimated. API cost ≈$0.37 per mix task; range $0.067–$2.81. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $1.13 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 89 configurations. Held-out mean absolute error 0.98 log dollars; benchmark-mean baseline 1.32. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈36.7; missing-input range 28.0–42.4 points. API cost ≈$3.31 per mix task; range $2.84–$4.02. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 59.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 13 training models, 38 configurations. Held-out mean absolute error 6.97 points; benchmark-mean baseline 11.99. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.36 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 13 training models, 38 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.77. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	55.06	≈$0.3658	≈36.72	≈$3.3057
GPT-6 Sol · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈38.2; missing-input range 28.9–47.4 points. API cost ≈$0.54 per mix task; range $0.31–$1.32. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 58.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.84 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	55.01	$0.0957	≈38.18	≈$0.5450
GPT-5.5 · low Evidence and eligibility Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark Figure estimate 70% of score weight measured. Score ≈54.9; missing-input range 46.5–54.9 points. API cost ≈$1.84 per mix task; range $0.36–$13.66. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 25.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 105 configurations. Held-out mean absolute error 13.09 points; benchmark-mean baseline 15.50. A higher or lower effort of the same model was measured, so the prediction is held at GPT-5.5 medium’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $5.65 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 88 configurations. Held-out mean absolute error 0.98 log dollars; benchmark-mean baseline 1.31. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 50% of score weight measured. Score ≈19.5; missing-input range 16.3–23.0 points. API cost ≈$0.82 per mix task; range $0.58–$1.88. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 3.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 36 training models, 103 configurations. Held-out mean absolute error 10.25 points; benchmark-mean baseline 15.40. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $0.83 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 30 training models, 81 configurations. Held-out mean absolute error 0.43 log dollars; benchmark-mean baseline 1.29. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 31.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 37 configurations. Held-out mean absolute error 5.50 points; benchmark-mean baseline 10.05. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.21 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 37 configurations. Held-out mean absolute error 0.60 log dollars; benchmark-mean baseline 0.87. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	≈54.94	≈$1.8437	≈19.48	≈$0.8237
GPT-5.6 Sol · low	54.24	$0.2424	28.66	$0.7105
GPT-5.6 Luna · xhigh	53.58	$0.0720	34.87	$0.7374
Claude Haiku 5.5 · low Evidence and eligibility Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure · Omniscience reliability: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Execute · AutomationBench-AA · partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published Figure estimate 100% of score weight measured. Score 53.1; no score component estimated. API cost ≈$0.056 per mix task; conditional tier envelope $0.019–$0.093. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · cost: $0.14 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.044 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.00080 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 60% of score weight measured. Score ≈35.8; missing-input range 26.5–45.0 points. API cost ≈$0.42 per mix task; range unknown, because part of the cost is an accounting scenario. Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 62.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.70 per scored attempt, estimated. Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $0.36 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.031 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.046 per benchmark task, estimated. Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08). Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.	53.11	≈$0.0561midpoint scenario · conditional $0.0187–$0.0935	≈35.76	≈$0.4156scenario + prediction · bill range unknown
GPT-6 Luna · high Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈33.0; missing-input range 22.8–43.1 points. API cost ≈$0.20 per mix task; range $0.074–$0.95. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 50.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.38 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	52.89	$0.0218	≈32.98	≈$0.2011
GPT-5.6 Terra · high	52.78	$0.3201	32.72	$0.7875
GLM-5.3 · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈52.2; missing-input range 48.0–52.6 points. API cost ≈$3.75 per mix task; range $3.07–$5.05. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 70.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 7.11 points; benchmark-mean baseline 12.19. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.60 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 13 training models, 39 configurations. Held-out mean absolute error 0.49 log dollars; benchmark-mean baseline 0.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	52.35	$0.2626	≈52.16	≈$3.7497
Qwen3.8 27B · medium Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈32.8; missing-input range 22.6–42.9 points. API cost ≈$3.50 per mix task; range $3.01–$4.29. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 51.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.38 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	52.25	$0.8645	≈32.78	≈$3.5026
Kimi K3 · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈39.1; missing-input range 30.6–43.3 points. API cost ≈$5.13 per mix task; range $4.31–$6.68. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 60.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 6.98 points; benchmark-mean baseline 12.22. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $4.31 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 13 training models, 39 configurations. Held-out mean absolute error 0.46 log dollars; benchmark-mean baseline 0.74. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	52.05	$0.2521	≈39.09	≈$5.1287
GPT-5.6 Luna · high	51.16	$0.0419	28.07	$0.3613
GPT-6 Luna · medium Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈27.5; missing-input range 17.4–37.7 points. API cost ≈$0.14 per mix task; range $0.048–$0.70. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 42.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.28 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	51.01	$0.0130	≈27.55	≈$0.1430
Qwen3.8 27B · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈26.5; missing-input range 16.3–36.7 points. API cost ≈$3.69 per mix task; range $3.02–$4.95. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 42.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $3.50 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	50.90	$0.6300	≈26.50	≈$3.6862
Gemini 3.5 Flash · medium Evidence and eligibility Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark Execute estimate 0% of score weight measured. Score ≈10.6; missing-input range 2.1–28.6 points. API cost ≈$7.10 per mix task; range $3.66–$13.93. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 13.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 8.20 points; benchmark-mean baseline 9.83. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $5.97 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality. 13 training models, 39 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.76. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · score: 0.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality. 36 training models, 105 configurations. Held-out mean absolute error 9.67 points; benchmark-mean baseline 15.32. The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $12.21 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality. 30 training models, 82 configurations. Held-out mean absolute error 0.47 log dollars; benchmark-mean baseline 1.26. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 28.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality, on a curve that levels off near the top. 36 training models, 106 configurations. Held-out mean absolute error 3.31 points; benchmark-mean baseline 8.09. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $3.17 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality. 32 training models, 89 configurations. Held-out mean absolute error 0.23 log dollars; benchmark-mean baseline 1.23. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 15.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality, on a curve that levels off near the top. 36 training models, 106 configurations. Held-out mean absolute error 8.01 points; benchmark-mean baseline 10.97. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.77 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Briefcase analytical quality. 31 training models, 84 configurations. Held-out mean absolute error 0.68 log dollars; benchmark-mean baseline 1.07. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	49.77	$2.5893	≈10.62	≈$7.0957
Gemini 3.5 Flash · high	49.42	$2.9098	26.55	$8.0699
Ling 3.0 Flash VL · unspecified Evidence and eligibility Figure excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Execute excluded from recommendations: Reasoning effort is not disclosed; cannot reproduce an exact setting. Figure · Briefcase analytical quality: Comparable API pricing is not published Figure · AA-LCR 1.1: Comparable API pricing is not published Figure · Omniscience reliability: Comparable API pricing is not published Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: Comparable API pricing is not published Execute · GDPval-AA v2.1: Comparable API pricing is not published Execute · AutomationBench-AA · partial credit: Comparable API pricing is not published Execute estimate 60% of score weight measured. Score ≈7.3; missing-input range 5.7–17.5 points. API cost unknown. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 4.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	48.63	Unpriced	≈7.33	Unpriced
GPT-6 Sol · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈38.4; missing-input range 29.2–47.6 points. API cost ≈$1.48 per mix task; range $1.00–$2.79. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 62.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.83 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	48.50	$0.1691	≈38.41	≈$1.4754
GPT-5.6 Terra · medium	48.26	$0.1746	23.26	$0.4181
GPT-5.5 · none Evidence and eligibility Execute excluded from recommendations: Only 10% of score weight measured; at least 50% required for a recommendation. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark Figure estimate 70% of score weight measured. Score ≈47.1; missing-input range 37.1–49.3 points. API cost ≈$1.82 per mix task; range $0.35–$13.48. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 29.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 105 configurations. Held-out mean absolute error 13.09 points; benchmark-mean baseline 15.50. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $5.58 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 88 configurations. Held-out mean absolute error 0.98 log dollars; benchmark-mean baseline 1.31. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 10% of score weight measured. Score ≈5.0; missing-input range 2.3–21.9 points. API cost ≈$0.55 per mix task; range $0.11–$3.17. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 1.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 6.66 points; benchmark-mean baseline 12.40. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.76 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 13 training models, 39 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 0.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · score: 1.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 36 training models, 103 configurations. Held-out mean absolute error 10.25 points; benchmark-mean baseline 15.40. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $0.62 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 30 training models, 81 configurations. Held-out mean absolute error 0.43 log dollars; benchmark-mean baseline 1.29. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 9.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 36 training models, 104 configurations. Held-out mean absolute error 7.76 points; benchmark-mean baseline 11.03. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.082 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 31 training models, 83 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.07. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	≈47.06	≈$1.8158	≈5.01	≈$0.5486
GPT-5.6 Terra · low	46.33	$0.1483	17.56	$0.3068
GPT-5.6 Sol · none Evidence and eligibility Execute excluded from recommendations: Only 25% of score weight measured; at least 50% required for a recommendation. Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute estimate 25% of score weight measured. Score ≈15.0; missing-input range 7.1–22.7 points. API cost ≈$0.84 per mix task; range $0.23–$4.39. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 15.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 13 training models, 35 configurations. Held-out mean absolute error 6.56 points; benchmark-mean baseline 12.06. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.02 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 13 training models, 38 configurations. Held-out mean absolute error 0.44 log dollars; benchmark-mean baseline 0.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · score: 5.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1, on a curve that levels off near the top. 36 training models, 101 configurations. Held-out mean absolute error 10.17 points; benchmark-mean baseline 15.53. A higher or lower effort of the same model was measured, so the prediction is held at Sol low’s measured score, loosened by the largest inversion measured on this benchmark between a non-reasoning and a reasoning setting (4.0 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $0.97 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: GDPval v2.1. 30 training models, 79 configurations. Held-out mean absolute error 0.42 log dollars; benchmark-mean baseline 1.29. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	45.72	$0.2520	≈14.97	≈$0.8358
GPT-5.6 Luna · medium	44.59	$0.0149	11.30	$0.1012
Qwen3.8 27B · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈6.4; missing-input range 4.7–16.6 points. API cost ≈$5.28 per mix task; range $4.43–$6.88. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 4.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $4.45 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	44.55	$2.7150	≈6.39	≈$5.2772
GPT-6 Luna · low Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈6.1; missing-input range 4.5–16.3 points. API cost ≈$0.044 per mix task; range $0.0077–$0.38. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 4.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.10 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	42.68	$0.0048	≈6.14	≈$0.0444
DeepSeek V4.1 Flash · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈34.4; missing-input range 24.2–44.5 points. API cost ≈$0.91 per mix task; range $0.55–$2.11. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 52.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $1.30 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	42.28	$0.0504	≈34.38	≈$0.9081
Gemini 3.5 Flash · minimal Evidence and eligibility Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark Figure estimate 70% of score weight measured. Score ≈41.8; missing-input range 32.9–41.8 points. API cost ≈$0.65 per mix task; range $0.12–$4.82. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 11.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 106 configurations. Held-out mean absolute error 12.82 points; benchmark-mean baseline 14.83. A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.5 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $2.00 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 89 configurations. Held-out mean absolute error 0.96 log dollars; benchmark-mean baseline 1.28. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 0% of score weight measured. Score ≈20.5; missing-input range 1.7–29.1 points. API cost ≈$2.25 per mix task; range $0.33–$19.88. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 21.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1, on a curve that levels off near the top. 13 training models, 42 configurations. Held-out mean absolute error 8.45 points; benchmark-mean baseline 10.88. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $2.28 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 13 training models, 42 configurations. Held-out mean absolute error 0.61 log dollars; benchmark-mean baseline 0.77. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · score: 9.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 105 configurations. Held-out mean absolute error 13.82 points; benchmark-mean baseline 15.32. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $3.43 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 30 training models, 83 configurations. Held-out mean absolute error 0.92 log dollars; benchmark-mean baseline 1.26. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 35.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 110 configurations. Held-out mean absolute error 7.06 points; benchmark-mean baseline 8.32. A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.5 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.0 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.81 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 32 training models, 94 configurations. Held-out mean absolute error 0.88 log dollars; benchmark-mean baseline 1.24. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 35.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 36 training models, 106 configurations. Held-out mean absolute error 10.07 points; benchmark-mean baseline 10.97. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.34 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: AA-LCR 1.1. 31 training models, 85 configurations. Held-out mean absolute error 0.39 log dollars; benchmark-mean baseline 1.07. Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.	≈41.84	≈$0.6485	≈20.47	≈$2.2454
GPT-5.6 Luna · low	39.48	$0.0097	4.85	$0.0365
GPT-5.6 Terra · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈13.0; missing-input range 5.0–15.6 points. API cost ≈$0.35 per mix task; range $0.18–$0.93. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 19.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 13 training models, 35 configurations. Held-out mean absolute error 6.79 points; benchmark-mean baseline 11.08. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.56 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 13 training models, 35 configurations. Held-out mean absolute error 0.47 log dollars; benchmark-mean baseline 0.69. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	38.11	$0.1518	≈12.98	≈$0.3473
DeepSeek V4 Pro 0813 · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈22.5; missing-input range 12.3–32.7 points. API cost ≈$1.82 per mix task; range $1.43–$2.52. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 34.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $2.15 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	37.39	$0.1468	≈22.51	≈$1.8181
GPT-6 Luna · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈18.6; missing-input range 8.4–28.7 points. API cost ≈$0.11 per mix task; range $0.033–$0.53. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 34.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.22 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 14 training models, 40 configurations. Held-out mean absolute error 0.45 log dollars; benchmark-mean baseline 0.73. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	33.29	$0.0091	≈18.58	≈$0.1057
GPT-5.6 Luna · none Evidence and eligibility Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute estimate 60% of score weight measured. Score ≈4.7; missing-input range 3.2–4.7 points. API cost ≈$0.18 per mix task; range $0.020–$2.72. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. DeepSWE v1.1 · mini-swe-agent · score: 3.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0, on a curve that levels off near the top. 13 training models, 35 configurations. Held-out mean absolute error 5.77 points; benchmark-mean baseline 9.75. A higher or lower effort of the same model was measured, so the prediction is held at Luna low’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (2.3 points). The held-out test does not establish accuracy here. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one. DeepSWE v1.1 · mini-swe-agent · cost: $0.43 per scored attempt, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: Terminal-Bench 4.0. 13 training models, 35 configurations. Held-out mean absolute error 0.43 log dollars; benchmark-mean baseline 0.63. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the lowest measured cost, so the prediction is held there. The held-out test does not establish accuracy here. Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.	28.82	$0.0098	≈4.73	≈$0.1804
GPT-5.4 Pro · xhigh Evidence and eligibility Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Token accounting is not published for this benchmark Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Token accounting is not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark	Unscored	Unpriced	Unscored	Unpriced
GPT-5.5 Pro · xhigh Evidence and eligibility Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Token accounting is not published for this benchmark Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Token accounting is not published for this benchmark Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Token accounting is not published for this benchmark Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1 Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Token accounting is not published for this benchmark	Unscored	Unpriced	Unscored	Unpriced
Claude Fable 5 · high Evidence and eligibility Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Artificial Analysis has not published a model row for this configuration Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Artificial Analysis has not published a model row for this configuration Figure estimate 0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$3.52 per mix task; range $1.81–$7.11. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 48.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $11.16 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · score: 82.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.51 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.053 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 40% of score weight measured. Score ≈50.6; missing-input range 42.5–56.7 points. API cost ≈$11.04 per mix task; range $7.03–$19.91. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 27.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $18.96 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.08. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 51.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 4.54 points; benchmark-mean baseline 7.10. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $4.69 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.02. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 55.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18. A higher or lower effort of the same model was measured, so the prediction is held at Fable 5 max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (1.7 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.74 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.88. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.	Unscored	≈$3.5216	≈50.55	≈$11.0392
Claude Fable 5 · low Evidence and eligibility Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Artificial Analysis has not published a model row for this configuration Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Artificial Analysis has not published a model row for this configuration Figure estimate 0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$1.22 per mix task; range $0.39–$3.89. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 41.6 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $3.79 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · score: 80.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.017 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 40% of score weight measured. Score ≈42.2; missing-input range 34.8–48.3 points. API cost ≈$3.79 per mix task; range $2.14–$9.83. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 16.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $5.77 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.08. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 47.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 4.54 points; benchmark-mean baseline 7.10. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $1.60 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.02. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 52.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.69 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.88. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.	Unscored	≈$1.2179	≈42.20	≈$3.7878
Claude Fable 5 · medium Evidence and eligibility Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Artificial Analysis has not published a model row for this configuration Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Artificial Analysis has not published a model row for this configuration Figure estimate 0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$2.16 per mix task; range $1.10–$4.39. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 46.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $6.79 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · score: 81.4 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.37 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.031 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 40% of score weight measured. Score ≈47.6; missing-input range 39.2–53.9 points. API cost ≈$6.73 per mix task; range $4.38–$11.95. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 23.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $10.98 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.08. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 49.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 4.54 points; benchmark-mean baseline 7.10. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $2.87 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.02. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 55.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18. A higher or lower effort of the same model was measured, so the prediction is held at Fable 5 max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (1.7 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $1.14 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.88. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.	Unscored	≈$2.1601	≈47.62	≈$6.7349
Claude Fable 5 · xhigh Evidence and eligibility Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Artificial Analysis has not published a model row for this configuration Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Artificial Analysis has not published a model row for this configuration Figure estimate 0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$4.77 per mix task; range $2.11–$11.12. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 49.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $15.17 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · score: 82.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.63 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.073 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 40% of score weight measured. Score ≈51.8; missing-input range 42.7–58.1 points. API cost ≈$13.09 per mix task; range $8.38–$25.33. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 29.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $19.29 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.08. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the highest measured cost, so the prediction is held there. The held-out test does not establish accuracy here. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 51.9 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 4.54 points; benchmark-mean baseline 7.10. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $6.37 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.02. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 55.8 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18. A higher or lower effort of the same model was measured, so the prediction is held at Fable 5 max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (1.7 points). The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $2.27 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.88. The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.	Unscored	≈$4.7710	≈51.78	≈$13.0942
Gemini 3.1 Pro Preview · high Evidence and eligibility Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Execute excluded from recommendations: Only 40% of score weight measured; at least 50% required for a recommendation. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Artificial Analysis has not published a model row for this configuration Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Artificial Analysis has not published a model row for this configuration Figure estimate 0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$0.63 per mix task; range $0.079–$4.96. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 4.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52. The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $1.92 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · score: 72.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.16 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.0085 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 40% of score weight measured. Score ≈10.5; missing-input range 7.6–17.0 points. API cost ≈$1.95 per mix task; range $1.05–$7.24. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 0.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45. The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $2.73 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.08. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 26.7 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 4.54 points; benchmark-mean baseline 7.10. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $0.82 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.02. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 21.2 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.39 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.88. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.	Unscored	≈$0.6269	≈10.54	≈$1.9536
Qwen3.8 Max · xhigh Evidence and eligibility Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation. Execute excluded from recommendations: Only 40% of score weight measured; at least 50% required for a recommendation. Figure · Briefcase analytical quality: No published measurement at this exact configuration Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration Figure · AA-LCR 1.1: No published measurement at this exact configuration Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration Figure · Omniscience reliability: No published measurement at this exact configuration Figure · Omniscience reliability: Artificial Analysis has not published a model row for this configuration Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration Execute · GDPval-AA v2.1: No published measurement at this exact configuration Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration Execute · AutomationBench-AA · partial credit: No published measurement at this exact configuration Execute · AutomationBench-AA · partial credit: Artificial Analysis has not published a model row for this configuration Figure estimate 0% of score weight measured. Score ≈unknown; missing-input range unknown–unknown points. API cost ≈$1.21 per mix task; range $0.39–$3.85. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Briefcase analytical quality · score: 40.0 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.90 points; benchmark-mean baseline 15.52. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Briefcase analytical quality · cost: $3.76 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 0.98. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · score: 80.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AA-Omniscience index · cost: $0.017 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 1.16. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Execute estimate 40% of score weight measured. Score ≈40.3; missing-input range 32.9–47.7 points. API cost ≈$3.75 per mix task; range $2.12–$9.74. Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling. Terminal-Bench 4.0 · score: 14.3 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. Terminal-Bench 4.0 · cost: $5.71 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.08. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · score: 46.5 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 4.54 points; benchmark-mean baseline 7.10. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. GDPval v2.1 · cost: $1.59 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 0.59 log dollars; benchmark-mean baseline 1.02. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · score: 51.1 converted points, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18. Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one. AutomationBench-AA partial credit · cost: $0.69 per benchmark task, estimated. Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval. Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.88. Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.	Unscored	≈$1.2069	≈40.29	≈$3.7534

Data: October 18, 2026 · Artificial Analysis, Datacurve · supporting: Cognition, ZapierPrices here compare setups against each other. Your actual bill will differ.

Scenario	Fixed · quality only	Fixed · priced	Expanded · quality only	Expanded · priced
Declared profile	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Elo slope 0.5×	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Elo slope 2×	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×0.5, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×0.5, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×0.5, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×1, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×1, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×1, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×2, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×2, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×0.5, reasoning ×2, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×0.5, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×0.5, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×0.5, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×1, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×1, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×1, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×2, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×2, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×1, reasoning ×2, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×2, reasoning ×0.5, reliability ×0.5	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models
Group weights analysis ×2, reasoning ×0.5, reliability ×1	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models
Group weights analysis ×2, reasoning ×0.5, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×2, reasoning ×1, reliability ×0.5	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models	Claude Opus 5.5 max104 configurations · 33 models
Group weights analysis ×2, reasoning ×1, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×2, reasoning ×1, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×2, reasoning ×2, reliability ×0.5	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×2, reasoning ×2, reliability ×1	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Group weights analysis ×2, reasoning ×2, reliability ×2	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Without briefcaseAnalysis	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Without lcr	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Without omniscience	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Without analysis	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Without reasoning	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models
Without reliability	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models	Claude Opus 5.5 max109 configurations · 33 models

Every configuration

See the trade-off

Benchmark frontiers

Models you can use

How this is measured

Evidence for Figure it out

Use your allowance well.

Settle the decision

Your agentClaude Opus 5.5 · max · Estimated

Use your allowance well.

Settle the decision

Claude CodeClaude Sonnet 5.5 · max · Estimated

What scores the Execute lane

Missing results and estimates

What this composition costs you

What the Figure lane's own sources are worth

Which models are included

The Execute leader and its limits

Supporting benchmarks and missing results

Where the numbers come from

Effort dials, model by model

What the prices do not include