Which model, at which effort?

Depends what the job is worth. Drag the line

Your job
Figure it outplanning & research
Claude Fable 5.2 max≈$18.21 / task · 81.1 pts
Execute itcode & commands
Kimi K3.1 Pro max≈$3.54 / task · ≈70.3 ptsKimi K3.1 Pro max≈70.3range 67.4–73.3Gemini 4 Carbon high≈69.7range 66.8–72.6The ranges overlap, so either could lead.

Figure it out. Price midway between list and fallback rates; the fallback's share isn't published. Under 60 points I wouldn't bother. The cheapest way over that bar: Muse Ember xhigh at ≈$0.038 (estimated).

Execute it. Close call: Kimi K3.1 Pro max ≈70.3 (67.4–73.3) against Gemini 4 Carbon high ≈69.7 (66.8–72.6); the ranges overlap. Score 60% measured, the rest predicted. Under 40 points I wouldn't bother. The cheapest way over that bar: GPT-6.1 Terra low at ≈$0.22 (estimated).

  • past redMuse Ember xhigh + GPT-6.1 Terra low18× cheaper
  • past amberGPT-6.1 Sol high + Muse Ember xhigh12× cheaper

Every configuration

quality against cost · open ↓close ↑

See the trade-off

Higher is better, left is cheaper. The dashed line joins the rungs on the rail above; the ring is the pick.

Benchmark frontiers

Figure score against cost per task, 132 configurations, log cost axis. Higher is better; left is cheaper.Scored on Briefcase analytical quality, AA-LCR 1.1, AA-Omniscience index; the flags mark the pick.
TOO WEAKJUST OK304050607080$0.01$0.10$1.00$10.00API dollars / benchmark mix task (USD, log scale)Figure scoreTHE PICK

Conditional tier envelope $0.019–$0.093 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.044–$0.22 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.072–$0.36 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.11–$0.57 per mix task; midpoint scenario, not expected price.

Conditional tier envelope $0.21–$1.05 per mix task; midpoint scenario, not expected price.

Claude Fable 5.2 · maxthe pick · ≈$18.21 / task · 81.08 pts

Chart key & sources
OpenAI
GPT-6.1 TerraGPT-6.1 SolGPT-6 LunaGPT-6 SolGPT-6 AstraGPT-5.6 LunaGPT-5.6 SolGPT-5.6 TerraGPT-5.5
Anthropic
Fable 5.2Haiku 5.5Sonnet 5.5Opus 5.5Fable 5.1Opus 5Fable 5
Moonshot
Kimi K3.1 ProKimi K3
DeepSeek
DeepSeek V4.1 Flash 1017DeepSeek V4.1 FlashDeepSeek V4 Flash VisionDeepSeek V4 Pro 0813
xAI
Grok 4.7Grok 4.6
Google
4 Carbon4 Argon3.8 Flash3.7 Flash3.5 Flash
Zhipu
GLM-5.3-FlashGLM-5.3
Alibaba
Qwen3.8 Max 0902Qwen3.8-Flash-NextQwen3.8 27B
Meta
Muse EmberMuse WonderMuse Spark 1.3Muse Spark 1.2
StepFun
Step 5 Preview
Xiaomi
MiMo-V2.6-FlashMiMo-V2.6-Pro
Mistral
Mistral Large 4
MiniMax
MiniMax M3.1 Flash
Thinking Machines
Inkling 2 Small
Tencent
Hy4
Hollow mark = estimated score or cost · pale dashed line = Pareto frontier · coloured line = one model's effort rungs · red = too weak, under the owner's 60-point floor · amber = just ok, 60–70 · the flags mark the pick · data: November 11, 2026

Models you can use

52 of 52 included · open ↓close ↑

Turn off what you can’t use; the rail recomputes. Open a model to drop single efforts.

6/6 efforts · edit
5/5 efforts · edit
6/6 efforts · edit
6/6 efforts · edit
6/6 efforts · edit
7/7 efforts · edit
7/7 efforts · edit
7/7 efforts · edit
5/5 efforts · edit
1/1 effort · edit
5/5 efforts · edit
5/5 efforts · edit
5/5 efforts · edit
5/5 efforts · edit
5/5 efforts · edit
5/5 efforts · edit
5/5 efforts · edit
1/1 effort · edit
3/3 efforts · edit
3/3 efforts · edit
2/2 efforts · edit
1/1 effort · edit
3/3 efforts · edit
3/3 efforts · edit
4/4 efforts · edit
3/3 efforts · edit
3/3 efforts · edit
3/3 efforts · edit
3/3 efforts · edit
3/3 efforts · edit
2/2 efforts · edit
1/1 effort · edit
2/2 efforts · edit
1/1 effort · edit
1/1 effort · edit
4/4 efforts · edit
1/1 effort · edit
2/2 efforts · edit
2/2 efforts · edit
2/2 efforts · edit
2/2 efforts · edit
1/1 effort · edit
1/1 effort · edit
1/1 effort · edit
3/3 efforts · edit
1/1 effort · edit
1/1 effort · edit
1/1 effort · edit
2/2 efforts · edit
1/1 effort · edit
1/1 effort · edit
1/1 effort · edit

How this is measured

sources, weights and limits · open ↓close ↑
How the rail is builtundominated rungs on a log job-share axis

The rail measures a whole job, not one benchmark task. Each dot is a model and effort that no cheaper comparable configuration beats on the mix score. A dot sits at what that rung contributes to the job: its lane's share of the job, times what the rung costs against the best rung in its own lane. The right end of the rail is the job with both lanes at their best, so the ticks read 1, .1, .01 — fractions of that job, not dollars. Equal distances still mean equal cost ratios.

Why not one dollar figure for both lanes? Because a benchmark task is not a job, and the two lanes' tasks are not the same size. Figure combines practical analysis, document reasoning and factual reliability. Execute combines repository work, terminal tasks, professional deliverables and app workflows. Their benchmark task sizes differ. Charging both lanes the same dollar ceiling treats one of each as an equal purchase — which it is not, when a real job runs one lane once and the other for hours.

"Your job is" sets the split. Mostly figuring puts 10% of the job in Execute, Balanced puts half, Mostly executing puts 90% there. The line is then one threshold both lanes' contributions sit under, and it converts back to a dollar ceiling per lane before anything is chosen — the lane carrying most of the job gets squeezed first, which is the point. Each answer still shows its own benchmark dollars per task and the date of the capture it came from.

Those three numbers are chosen, not measured. They are round positions, with the two "mostly" options symmetric around Balanced. There is no retained source behind 10, 50 and 90, and none is claimed: they exist so a reader can say roughly what their week looks like, and a spinner would promise a precision nobody here has. Pick the one nearest your own and read the answer it gives.

The author's own jobs are a separate thing, and they are measured. Across 16 of his Claude Code jobs that ran both a figuring phase and an executing phase, 2026-09-03 to 09-08, executing took 5 to 94 percent of those two phases' API-equivalent cost at that day's published rates — median 84 percent, 75 percent weighted by dollars, over $347 of attributed spend. The largest such job spent $63 of $77 executing. That is one person's five days on one harness. It is not a bill, it is not a measurement of your work, and the width of that range is exactly why the split is a control rather than a constant.

Those runs were also almost all cache reads: 98.4 percent of input-side tokens across the Claude Code sessions, and 96.1 percent of input tokens across 258 Codex sessions in the same window. For work shaped like that a model's cache-read price orders the bill far more than its input price does, and a benchmark task bounded at a few minutes cannot show you that. Opus 5 reads cache at $0.50 per million tokens, Fable 5.1 at $0.25, Sonnet 5 at $0.20, Haiku 4.5 at $0.10 (Anthropic pricing, read 2026-09-08).

The rail carries the undominated rungs only. Cheaper and stronger steps are the dots themselves: every Pareto step for both lanes is on the line, and dragging to one selects it. A ceiling below every rung produces no recommendation for that lane and names the cheapest comparable option instead.

How alternatives qualifyscreening rules and evidence limits

A model does not need every benchmark to earn consideration. One strong, relevant evaluation can qualify a scoped alternative. Figure looks across practical analysis, reasoning, factual reliability and document work. Execute looks across coding-agent work, professional outputs and tool workflows. A specialist remains a specialist; its best result does not establish broad capability.

Within your available models and effort settings, each benchmark admits results within 5 score points of its leader, or 100 Elo on native Elo boards. It also admits undominated quality–cost options within 20 points, or 400 Elo, of the leader. “Undominated” means no measured option is both at least as good and at least as cheap, with one strict advantage. Missing cost cannot earn a value claim. These margins are chosen screening rules, not uncertainty estimates or proof of a meaningful lead.

Each track shows its mix and relevant supporting evidence. Figure also carries hard-question and document-reading results; Execute carries FrontierCode, Zapier, SciCode, Briefcase rubric checks and other agent-work results. Supporting scores do not enter the mix. Every qualifying model remains accessible; the first six are a compact alphabetical preview. Open a model's row to see efforts, native scores, source, date, harness and fallback status. Several results from one publisher are not independent corroboration.

The shortlist uses native Briefcase and GDPval Elo and the published AA-Omniscience index. Both mixes use the fixed conversions below; the native evidence stays visible beside them. Native and composite units must not be compared directly. FrontierCode does not say whether Claude Code rerouted a Claude run to a fallback model. Its Claude results count only for models Claude Code never reroutes. Kimi's Zapier effort is undisclosed. Such scoped results can support consideration, but cannot fill an exact mix gap.

How the recommendation is pickedquality, spending and uncertainty

Start with quality, then decide how much to spend. Quality first selects the highest score in the declared mix among configurations with enough measured or explicitly estimated inputs. When a source does not name the effort it ran, the row is read as the vendor's default, the setting you get by naming none, and every surface labels it default (assumed). Equal scores prefer lower cost, then a stable configuration identity. Estimates can keep useful models in the comparison, but when the pick or an alternative under the same ceiling is estimated and their ranges overlap, the answer is marked close call, which opens a note naming both configurations. The range does not include all uncertainty in the measured benchmarks.

There is one mix for Figure it out and one for Execute it. The provider chips filter models you can access: Codex is OpenAI-only; Claude Code is Anthropic-only. Each benchmark keeps the harness that measured it. The composite does not claim to measure the model inside either product.

The rail sets a logarithmic ceiling in shares of one job: equal distances mean equal cost ratios, and the position converts to a dollar ceiling per lane through that lane's share of the job. Each kind of work then answers with its highest score within its own ceiling. Every Pareto step is a dot on the rail, so dragging past one selects it. A ceiling too low produces an explicit empty recommendation with the cheapest available alternative. Releasing the rail at its top removes the ceiling — Quality first — without changing model or effort exclusions.

The floors are the author's judgment, not a measurement. Below 60 points for Figure it out, or 40 for Execute it, he would not run that lane at all; from there up to 70 and 50 a configuration is acceptable but not strong. The rail and the Every configuration chart shade those stretches red and amber, and an answer that lands in one says so. The floors never change the pick — it is still the highest score under your ceiling — and no source stands behind the numbers.

Cheaper models that are not on the frontier still appear where their evidence earns it: the shortlist under Why this choice states each qualifying model’s measured scope, system, and unknown or above-ceiling costs. A component strength is not a broad recommendation.

The audit tests the policy, not the probability of being right. Open Method sensitivity & winning margins to see whether the pick survives half or double Elo slopes, wider group weights, or a benchmark left out. Estimates stay fixed while the weights move, and scenario counts are not confidence percentages. A stable result can still be wrong for your task: the scenarios cannot supply real-world outcomes that were never measured. The Execute leader and its limits explains what the Execute pick with no budget limit rests on, for All models and Claude Code.

Why this choicemeasurements, method & access

132 of 145 configurations can set this lane’s recommendation, 65 of them with estimates.

Why 13 configurations cannot set this recommendation
  • Gemini 3.7 Flash low · Benchmark-specific harnesses

    ≈54.10 points (estimated) · ≈$0.54 / profile task (estimated)

    Only 30% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Token accounting is not published for this benchmark
    • AA-LCR 1.1: Comparable API pricing is not published
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
    How the estimate is made

    30% of score weight measured. Score ≈54.1; missing-input range 37.2–62.1 points. API cost ≈$0.54 per mix task; range $0.063–$4.70.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 25.7 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 42 training models, 123 configurations. Held-out mean absolute error 12.64 points; benchmark-mean baseline 14.62.

      A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.7 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $1.65 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.14 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 0.95.

      Missing measurement: Comparable API pricing is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 57.0 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.0077 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Gemini 3.7 Flash medium · Benchmark-specific harnesses

    ≈56.48 points (estimated) · ≈$0.61 / profile task (estimated)

    Only 30% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Token accounting is not published for this benchmark
    • AA-LCR 1.1: Comparable API pricing is not published
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
    How the estimate is made

    30% of score weight measured. Score ≈56.5; missing-input range 39.3–65.7 points. API cost ≈$0.61 per mix task; range $0.073–$5.08.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 25.7 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 42 training models, 123 configurations. Held-out mean absolute error 12.64 points; benchmark-mean baseline 14.62.

      A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.7 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $1.85 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.16 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 0.95.

      Missing measurement: Comparable API pricing is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 59.7 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.0084 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GPT-5.5 low · Benchmark-specific harnesses

    ≈55.30 points (estimated) · ≈$1.82 / profile task (estimated)

    Only 30% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Token accounting is not published for this benchmark
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
    How the estimate is made

    30% of score weight measured. Score ≈55.3; missing-input range 36.9–65.0 points. API cost ≈$1.82 per mix task; range $0.38–$12.02.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 25.4 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 42 training models, 121 configurations. Held-out mean absolute error 12.68 points; benchmark-mean baseline 14.76.

      A higher or lower effort of the same model was measured, so the prediction is held at GPT-5.5 medium’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $5.56 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 36 training models, 98 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.37.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 58.5 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.023 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GPT-5.5 Pro xhigh · Benchmark-specific harnesses

    Score unavailable · Comparable cost unavailable

    Only 0% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Token accounting is not published for this benchmark
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Token accounting is not published for this benchmark
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
  • Gemini 3.5 Flash minimal · Benchmark-specific harnesses

    ≈40.18 points (estimated) · ≈$0.68 / profile task (estimated)

    Only 30% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Token accounting is not published for this benchmark
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
    How the estimate is made

    30% of score weight measured. Score ≈40.2; missing-input range 23.2–48.2 points. API cost ≈$0.68 per mix task; range $0.13–$4.98.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 11.0 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 42 training models, 122 configurations. Held-out mean absolute error 12.52 points; benchmark-mean baseline 14.16.

      A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.5 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $2.09 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 36 training models, 99 configurations. Held-out mean absolute error 1.00 log dollars; benchmark-mean baseline 1.34.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 46.2 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.0096 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GPT-5.5 none · Benchmark-specific harnesses

    ≈47.88 points (estimated) · ≈$1.80 / profile task (estimated)

    Only 30% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Token accounting is not published for this benchmark
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
    How the estimate is made

    30% of score weight measured. Score ≈47.9; missing-input range 29.5–57.8 points. API cost ≈$1.80 per mix task; range $0.38–$11.87.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 31.8 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 42 training models, 121 configurations. Held-out mean absolute error 12.68 points; benchmark-mean baseline 14.76.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $5.49 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 36 training models, 98 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.37.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 47.8 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.023 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Ling 3.0 Flash VL default (assumed) · Benchmark-specific harnesses

    ≈52.26 points (estimated) · Comparable cost unavailable

    • Briefcase analytical quality: Comparable API pricing is not published
    • AA-LCR 1.1: Comparable API pricing is not published
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Token accounting is not published for this benchmark
    How the estimate is made

    60% of score weight measured. Score ≈52.3; missing-input range 42.5–62.0 points. API cost unknown.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • AA-Omniscience index · score: 56.8 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Claude Fable 5 high · Benchmark-specific harnesses

    ≈62.14 points (estimated) · ≈$3.45 / profile task (estimated)

    Fallback behavior is not disclosed; exact system cannot be established

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
    How the estimate is made

    0% of score weight measured. Score ≈62.1; missing-input range 46.4–73.1 points. API cost ≈$3.45 per mix task; range $1.31–$9.12.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 48.4 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $10.95 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · score: 82.0 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.51 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 57.5 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.033 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Claude Fable 5 low · Benchmark-specific harnesses

    ≈56.02 points (estimated) · ≈$1.23 / profile task (estimated)

    Fallback behavior is not disclosed; exact system cannot be established

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
    How the estimate is made

    0% of score weight measured. Score ≈56.0; missing-input range 41.7–68.7 points. API cost ≈$1.23 per mix task; range $0.46–$3.40.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 41.3 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $3.83 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · score: 80.4 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 48.7 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.015 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Claude Fable 5 medium · Benchmark-specific harnesses

    ≈59.83 points (estimated) · ≈$2.15 / profile task (estimated)

    Fallback behavior is not disclosed; exact system cannot be established

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
    How the estimate is made

    0% of score weight measured. Score ≈59.8; missing-input range 44.3–71.7 points. API cost ≈$2.15 per mix task; range $0.93–$5.06.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 45.9 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $6.76 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · score: 81.4 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.36 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 54.1 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.023 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Claude Fable 5 xhigh · Benchmark-specific harnesses

    ≈63.11 points (estimated) · ≈$4.64 / profile task (estimated)

    Fallback behavior is not disclosed; exact system cannot be established

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
    How the estimate is made

    0% of score weight measured. Score ≈63.1; missing-input range 47.4–73.7 points. API cost ≈$4.64 per mix task; range $1.52–$14.16.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 49.5 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $14.78 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · score: 82.2 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.63 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

      The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 59.0 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.041 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Gemini 3.1 Pro Preview high · Benchmark-specific harnesses

    ≈39.67 points (estimated) · ≈$0.64 / profile task (estimated)

    Only 0% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
    How the estimate is made

    0% of score weight measured. Score ≈39.7; missing-input range 28.6–50.8 points. API cost ≈$0.64 per mix task; range $0.077–$5.42.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 4.7 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

      The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $1.98 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · score: 72.3 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.16 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 41.5 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

      The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.0089 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Qwen3.8 Max xhigh · Benchmark-specific harnesses

    ≈54.72 points (estimated) · ≈$1.22 / profile task (estimated)

    Only 0% of score weight measured; at least 50% required for a recommendation

    • Briefcase analytical quality: No published measurement at this exact configuration
    • Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
    • AA-LCR 1.1: No published measurement at this exact configuration
    • AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
    • AA-Omniscience index: No published measurement at this exact configuration
    • AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
    How the estimate is made

    0% of score weight measured. Score ≈54.7; missing-input range 40.4–69.1 points. API cost ≈$1.22 per mix task; range $0.45–$3.37.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · score: 39.7 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Briefcase analytical quality · cost: $3.80 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · score: 80.1 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.24 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · score: 47.0 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-Omniscience index · cost: $0.015 per benchmark task, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

      Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Evidence behind the chosen configuration

The weights are declared policy, not a validated prediction of your work. Collaborative framing and planning remain unmeasured.

Score & native measurements

81.08 points · Figure it out. Benchmark proxies do not guarantee correctness.

BenchmarkNative resultAPI $ / task
Briefcase analytical quality30.0% weight · AA benchmark harnessClaude Fable 5.2 (Max, Default Fallback)2015.01 Elo$59.35
AA-LCR 1.130.0% weight · AA benchmark harnessClaude Fable 5.2 (Max, Default Fallback)83.33%≈$1.05 (estimated)
AA-Omniscience index40.0% weight · AA benchmark harnessClaude Fable 5.2 (Max, Default Fallback)66.77 points$0.22

Supporting evidence, outside the score: Briefcase presentation 1706.03 Elo · 2026-11-11. Same exact model and effort; AA benchmark harness.

How the estimate is made

100% of score weight measured. Score 81.1; no score component estimated. API cost ≈$18.21 per mix task; range $18.13–$18.28.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • AA-LCR 1.1 · cost: $1.05 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Claude Haiku 5.5 max Figure: $0.21–$1.05 per mix task, a conditional tier envelope. The rail uses its $0.63 midpoint scenario, not an expected price.

At this ceiling: lower-tier scenario → Claude Fable 5.2 max; midpoint → Claude Fable 5.2 max; upper-tier scenario → Claude Fable 5.2 max. The budget choice stays the same at these three scenarios.

Tier assumptions and all measured efforts

Only request-tier allocation varies, using captured AA tokens, cache-hit rate, five-minute cache writes and task counts. Every request above 100,000 prompt tokens, including cached tokens, costs five times as much. This envelope does not guarantee an actual bill or establish a confidence interval. Other billing modifiers are outside it. Estimates may exceed your ceiling. All tier rows move together in these three readings; they do not cover every independent allocation.

  • Claude Haiku 5.5 low: $0.019–$0.093 per Figure mix task; 52.9 points measured.
  • Claude Haiku 5.5 medium: $0.044–$0.22 per Figure mix task; 60.3 points measured.
  • Claude Haiku 5.5 high: $0.072–$0.36 per Figure mix task; 62.1 points measured.
  • Claude Haiku 5.5 xhigh: $0.11–$0.57 per Figure mix task; 64.5 points measured.
  • Claude Haiku 5.5 max: $0.21–$1.05 per Figure mix task; 68.1 points measured.

Execute predicts its missing DeepSWE component separately; its total bill has no established bound.

Access: subscription, API or your hardware

Reviewed 2026-11-11. Availability does not reproduce a benchmark’s agent, effort or fallback configuration.

Subscription Verified route

Max, Ultra, premium Team and premium legacy seat-based Enterprise seats include Fable models within up to 50% of the shared weekly allowance. Pro, standard Team and standard Enterprise seats use purchased usage credits from the start. A launch offer includes Fable on Pro and Team standard seats through 2026-11-16 23:59 PT.

Plan-dependent inclusion or credits; the 50% is not extra capacity. Usage-based Enterprise uses API rates. Launch-week Pro inclusion ends 16 November 2026.

Claude Code requires 2.1.321 or newer. App fallback is automatic; availability alone does not reproduce the benchmark fallback system. Mythos 5.2 shares the same specs and price but is limited-availability verification-program access and is not this model.

Fable plan rules ↗ · Fable 5.2 launch offers ↗ · Fallback behavior ↗

API Verified route

claude-fable-5-2, released 2026-11-09. Adaptive thinking always on; default effort high.

$8 input / $40 output USD per million tokens; 5-minute cache write $10, 1-hour $16, cache read $0.20. Batch is 50% off. The $16 / $80 fast setting is not the measured configuration.

API fallback must be configured. Cost and quality depend on the exact fallback model/settings, not only the primary model. Effort: low through max; default high. Mythos 5.2 is a separate limited-availability id. The $16 / $80 fast setting is a different price and is unmeasured.

API pricing ↗ · Fable 5.2 overview ↗ · Fallback behavior ↗

Self-hosting Not verified

No verified open-weight or self-hosting route for this exact model.

Not verified

Do not infer exact-version availability or included usage from a family name.

API pricing ↗ · Fable 5.2 overview ↗

Method sensitivity & winning margins

These are declared scenarios, not confidence percentages. They vary Elo slope by half/double, group weights by 0.5/1/2, omit benchmarks/groups. Your setup and exclusions apply. Priced results apply your spending ceiling; quality-only results ignore costs.

Fixed cohorts keep the baseline eligible configurations; expanded cohorts admit configurations newly scoreable under each composition. Validated estimates stay fixed while weights vary. Native margins below include only measured pairs; these scenarios do not test estimation uncertainty. Several components share Artificial Analysis, model judges and agent tools; they are not independent confirmations. Benchmark-specific harness and fallback limits remain.

Baseline: 132 priced configurations from 45 models. Claude Fable 5.2 max leads Claude Opus 5.5 max by 1.668 points.

  • Briefcase analytical quality: native margin -110.5800 at 30.0% weight.
  • AA-LCR 1.1: native margin -0.0133 at 30.0% weight.
  • AA-Omniscience index: native margin +18.6333 at 40.0% weight.
ScenarioFixed · quality onlyFixed · pricedExpanded · quality onlyExpanded · priced
Declared profileClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Elo slope 0.5×Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Elo slope 2×Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Group weights: practical analysis ×0.5, document reasoning ×0.5, factual reliability ×1Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Group weights: practical analysis ×0.5, document reasoning ×0.5, factual reliability ×2Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Group weights: practical analysis ×0.5, document reasoning ×1, factual reliability ×0.5Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Group weights: practical analysis ×0.5, document reasoning ×1, factual reliability ×1Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Group weights: practical analysis ×0.5, document reasoning ×1, factual reliability ×2Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Group weights: practical analysis ×0.5, document reasoning ×2, factual reliability ×0.5Claude Fable 5.2 max124 configurations · 43 modelsClaude Fable 5.2 max123 configurations · 42 modelsClaude Fable 5.2 max129 configurations · 43 modelsClaude Fable 5.2 max128 configurations · 42 models
Group weights: practical analysis ×0.5, document reasoning ×2, factual reliability ×1Claude Fable 5.2 max124 configurations · 43 modelsClaude Fable 5.2 max123 configurations · 42 modelsClaude Fable 5.2 max129 configurations · 43 modelsClaude Fable 5.2 max128 configurations · 42 models
Group weights: practical analysis ×0.5, document reasoning ×2, factual reliability ×2Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Group weights: practical analysis ×1, document reasoning ×0.5, factual reliability ×0.5Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Group weights: practical analysis ×1, document reasoning ×0.5, factual reliability ×1Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Group weights: practical analysis ×1, document reasoning ×0.5, factual reliability ×2Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Group weights: practical analysis ×1, document reasoning ×1, factual reliability ×0.5Claude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 modelsClaude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 models
Group weights: practical analysis ×1, document reasoning ×2, factual reliability ×0.5Claude Opus 5.5 max124 configurations · 43 modelsClaude Opus 5.5 max123 configurations · 42 modelsClaude Opus 5.5 max129 configurations · 43 modelsClaude Opus 5.5 max128 configurations · 42 models
Group weights: practical analysis ×2, document reasoning ×0.5, factual reliability ×0.5Claude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 modelsClaude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 models
Group weights: practical analysis ×2, document reasoning ×0.5, factual reliability ×1Claude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 modelsClaude Fable 5.2 max133 configurations · 46 modelsClaude Fable 5.2 max132 configurations · 45 models
Group weights: practical analysis ×2, document reasoning ×0.5, factual reliability ×2Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Group weights: practical analysis ×2, document reasoning ×1, factual reliability ×0.5Claude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 modelsClaude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 models
Group weights: practical analysis ×2, document reasoning ×2, factual reliability ×0.5Claude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 modelsClaude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 models
Without Briefcase analytical qualityClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Without AA-LCR 1.1Claude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 modelsClaude Fable 5.2 max97 configurations · 37 models
Without AA-Omniscience indexClaude Opus 5.5 max133 configurations · 46 modelsClaude Opus 5.5 max132 configurations · 45 modelsClaude Opus 5.5 max138 configurations · 46 modelsClaude Opus 5.5 max137 configurations · 45 models

Evidence for Figure it out

27 models

Relevant benchmark strengths, including setups that cannot set a priced recommendation. Supporting benchmarks stay separate from the profile score.

Claude Fable 5.1Scientific reasoning · Reading documents +2Comparable profile option · One publisher

At least one qualifying effort has complete, priced profile evidence.

Scientific reasoning · Reading documents · Hard reasoning · Long-context reasoning

  • CritPt ↗xhigh effort · 31.14 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • GDP.pdf all-pass ↗low effort · 28.00 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • HLE ↗xhigh effort · 58.71 points · $1.03 / benchmark taskQuality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • HLE ↗max effort · 59.13 points · $1.59 / benchmark taskQuality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-LCR 1.1 ↗max effort · 85.33 points · $1.58 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-LCR 1.1 ↗high effort · 83.67 points · $1.48 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-LCR 1.1 ↗medium effort · 84.67 points · $1.48 / benchmark taskNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
Claude Fable 5.2Practical analysis · Scientific reasoning +2Comparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. Anthropic API model claude-fable-5-2, released November 9, adaptive thinking always on, default effort high. AA measures low through max with Default Fallback. Costs retain AA's captured $8 / $40 list rate (cache-read $0.20). The $16 / $80 fast setting and Mythos 5.2 are not this configuration. Claude Code requires 2.1.321 or newer. Direct DeepSWE is unmeasured.

Practical analysis · Scientific reasoning · Hard reasoning · Factual reliability

  • Briefcase analytical quality ↗max effort · 2015.01 Elo · $59.35 / benchmark taskQuality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗max effort · 35.43 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗xhigh effort · 35.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗high effort · 32.86 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗medium effort · 30.86 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • HLE ↗max effort · 74.28 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-Omniscience index ↗max effort · 66.77 points · $0.22 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-Omniscience index ↗xhigh effort · 64.77 points · $0.033 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-Omniscience index ↗high effort · 64.03 points · $0.020 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-Omniscience index ↗medium effort · 63.50 points · $0.015 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-Omniscience index ↗low effort · 62.93 points · $0.014 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · fallback enabled · 2026-11-11
Claude Haiku 5.5Financial analysisComparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA now measures low through max as standalone efforts, matching the documented absence of server-side fallback. Lower-tier tariffs are restored, but the exact-row caveat says 5x long-context pricing is omitted. Aggregate tokens do not establish request-tier allocations; native benchmark bills stay missing. Figure uses a separately marked conditional tier envelope and non-expected midpoint; Execute predicts missing DeepSWE separately and has no established total-bill bound. AA flags pre-release AutomationBench refusals pending reevaluation. Direct DeepSWE is unmeasured; fallback-unknown Claude Code and fallback-labelled Zapier results join nothing.

  • AA-AnalystAgent pass@1 ↗max effort · 68.25 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · single model · 2026-11-11
Claude Opus 5.5Practical analysis · Scientific reasoning +2Comparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA measures low through max with default safety fallback. AA publishes a fallback tariff for every effort and, among the benchmarks used here, fallback fractions for AA-Omniscience only: its cost is a direct bill, and every other Opus 5.5 AA cost is a cost bound shown at its midpoint. No matching direct DeepSWE result exists; its cost is predicted from the AA-Omniscience bill.

Practical analysis · Scientific reasoning · Reading documents · Long-context reasoning

  • Briefcase analytical quality ↗max effort · 2125.59 Elo · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • Briefcase analytical quality ↗xhigh effort · 2080.87 Elo · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗max effort · 31.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗xhigh effort · 31.71 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗high effort · 30.86 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • GDP.pdf all-pass ↗high effort · 28.80 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-LCR 1.1 ↗max effort · 84.67 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-LCR 1.1 ↗xhigh effort · 84.67 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • AA-LCR 1.1 ↗medium effort · 84.33 points · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
Claude Sonnet 5.5Financial analysis · Practical analysis +1Comparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA measures low through max with default safety fallback, which Anthropic routes to Sonnet 5, whose input/output list prices match. AA captures the $0.10 cache-read rate, reduced October 7. AA publishes no fallback route or fractions, so costs are primary-tariff scenarios. FrontierCode Claude Code runs do not disclose fallback use and join nothing. No matching direct DeepSWE result exists.

Financial analysis · Practical analysis · Scientific reasoning

  • AA-AnalystAgent pass@1 ↗max effort · 69.50 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • Briefcase analytical quality ↗max effort · 2100.10 Elo · Fallback token fractions are not published for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗max effort · 31.43 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
  • CritPt ↗xhigh effort · 31.14 points · Comparable task accounting is not captured for this benchmarkNear the available leader · AA benchmark harness · fallback enabled · 2026-11-11
DeepSeek V4.1 FlashLong-context reasoningComparable profile option · One publisher · Effort / access caveat

At least one qualifying effort has complete, priced profile evidence. AA measured max effort and thinking disabled on the September release. Other native reasoning settings are unmeasured; earlier V4 Flash and V4 Pro measurements do not transfer, and neither do these to DeepSeek-V4.1-Flash-1017, which the deepseek-flash alias has served since 2026-10-17. AA prints that build as separate max, high and non-reasoning rows, stored as DS V4.1 Flash 1017.

  • AA-LCR 1.1 ↗max effort · 84.00 points · $0.031 / benchmark taskNear the available leader · Quality–cost frontier · AA benchmark harness · single model · 2026-11-11
How options qualify & what this search misses

The screening rules are under How alternatives qualify. No cost means no value claim, and a strength is not an endorsement of the whole model.

On 2026-11-11 this search examined 734 AA configurations, 134 FrontierCode entries and 134 Zapier entries. Coverage and public availability remain incomplete; this is a source-backed shortlist, not proof that every great model was found.

Access reference for all 52 maintained models
Access: subscription, API or your hardware

Reviewed 2026-09-10. Availability does not reproduce a benchmark’s agent, effort or fallback configuration.

Subscription Verified route

Plus and Business Standard include limited Astra usage; Pro and Business Premium can use their full existing Work/Codex allowance on Astra.

Plan subscription plus optional credits; task/model-dependent allowance.

Work and Codex share usage; Chat has separate allowances. Optional purchased credits extend usage. Inclusion is not unlimited usage or zero API task cost.

Codex plans ↗ · Work and Codex plans ↗

API Verified route

gpt-6-astra

$10 input / $50 output USD per million tokens; standard API billing.

API efforts: low, medium, high, xhigh, max. Harness menus may differ. Long-context surcharges and caching terms are on the model page. Ultra is a separate multiagent system; benchmark scores do not transfer to it.

API model ↗

Self-hosting Not verified

No verified open-weight or self-hosting route for this exact model.

Not verified

Do not infer exact-version availability or included usage from a family name.

API model ↗

Use your allowance wellchat, agent and when to switch

Practical guidance for each kind of work, from the configuration the rail names. There is no measured quota optimum.

Use your allowance well.

  1. Settle the decision

    Decide what success means with the model that has the useful context.

  2. Your agentClaude Fable 5.2 · max · Estimated

    Start here when the investigation needs files, tests or tools.

Limits & when to switch

On Claude Pro and Max, Chat and Claude Code share usage limits. Switching between them adds no separate allowance. Max and Team include monthly Claude Console API credits, which do not cover Claude Code or app extra usage or change those limits.

Skip a separate chat when the plan is settled or the decision needs repository tools. Reconsider the model or effort when the design is unresolved or fixes keep failing; carry the useful context with you.

Move the rail to inspect the trade-off. API savings do not measure subscription savings, and the benchmarks do not identify when to escalate.

Selected model access: Max, Ultra, premium Team and premium legacy seat-based Enterprise seats include Fable models within up to 50% of the shared weekly allowance. Pro, standard Team and standard Enterprise seats use purchased usage credits from the start. A launch offer includes Fable on Pro and Team standard seats through 2026-11-16 23:59 PT. Check access & evidence below.

Product facts checked 2026-10-08. Opus 5.5 model accessClaude Pro and Max usageMax and Team API credits

Practical guidance · no measured quota optimum.

Use your allowance well.

  1. Settle the decision

    Decide what success means with the model that has the useful context.

  2. Your agentKimi K3.1 Pro · max · Estimated

    Bring the decision. Build, verify and fix in the same task.

Limits & when to switch

Check whether your chat and agent tools share an allowance before moving work between them.

Skip a separate chat when the plan is settled or the decision needs repository tools. Reconsider the model or effort when the design is unresolved or fixes keep failing; carry the useful context with you.

Move the rail to inspect the trade-off. API savings do not measure subscription savings, and the benchmarks do not identify when to escalate.

Selected model access: Kimi Code: k3.1-pro on Moderato / Plus plans and above; up to 1M context on Allegretto / Pro and above. Check access & evidence below.

Product facts checked 2026-10-08.

Practical guidance · no measured quota optimum.

Subscription, API or self-hostingwhat the cost numbers mean

Benchmark API cost is a comparison unit, not your bill. Subscription plans may include an allowance, require credits from the first request, restrict supported tools or leave exact versions undisclosed. Max and Team now include monthly Claude Console API credits; these do not cover Claude Code or app extra usage or change their limits. The selected model’s access disclosure and the full access reference keep those distinctions separate, with official sources and verification dates.

OpenAI’s October system card distinguishes October Chat Sol/Luna checkpoints from September versions in Work and Codex. The Chat variants do not inherit the measurements here; official help and launch pages also disagree about Chat availability and rollout scope.

Downloadable weights establish a self-hosting route, not laptop suitability or savings. The reference gives sourced serving examples and distinguishes author formats, community quantizations and related hosted variants. For example, hosted Qwen Flash is based on Flash-Next with additional production features; it is not silently treated as the same measured weights configuration. Local hardware, electricity and operations costs are not estimated here.

What turning the effort dial up costsmeasured steps and regressions

Turning the dial up buys the model more thinking. You pay for that thinking in tokens, so it is a spending dial first and a quality dial second.

That makes it measurable. Every row here is one model at one setting, scored on DeepSWE v1.1 · mini-swe-agent, Terminal-Bench 4.0 · AA harness, GDPval-AA v2.1 and AutomationBench-AA partial credit and billed with the same job weights. The ladder compares complete measured settings; a missing setting is skipped, not estimated.

GPT-5.6 Terra shows it most starkly. Among models with at least four complete settings, the last step costs about 10× what the first did: $0.020 a point at the bottom, $0.20 at the top. Measured November 11, 2026.

GPT-5.6 Terra — each step of the reasoning dial, on the execution lane
StepScorePoints boughtAdded cost$ / point
low → medium23.30+5.70$0.1113$0.0195
medium → high32.77+9.47$0.3694$0.0390
high → xhigh39.45+6.68$0.7733$0.1158
xhigh → max53.98+14.53$2.8868$0.1987

No number here tells you where to stop. The board prices a point at each step. What a wrong answer costs you is the other half, and only you have it.

Higher does not always mean better, either. 2 measured configurations score lower at a higher setting:

  • GPT-6 Astra · xhigh → max — 0.25 points down, $2.33 more per task. Paying more for a worse result.
  • Grok 4.6 · high → xhigh — 0.60 points down, $0.76 more per task. Paying more for a worse result.

These measured component scores do not include benchmark error bars. Read that as a reason to check your own setting, not as a law about the model. Either way, “turn it to max” is a guess rather than advice.

Never pick the setting on its own. A cheap model turned up and a costly one turned down are the same kind of thing here — which is why the card compares configurations rather than ranking model names.

What each task is measured onsources, weights and missing evidence

Two owned mixes define the work. Their weights are reasoned policy from a two-week sample of the author's own agent work (213 sessions, 2026-09-03 to 2026-09-16), changed as little as that sample allowed. They are not scientifically established constants. Publisher composite indexes cannot change these formulas or decide which effort settings appear.

  • Figure it out: 40% AA-Omniscience, 30% Briefcase analytical quality, 30% AA-LCR v1.1.
  • Execute it: 40% direct DeepSWE v1.1, 35% AA Terminal-Bench 4.0, 15% AutomationBench-AA partial credit, 10% GDPval-AA v2.1.

What scores the Execute lane

Direct DeepSWE v1.1 measures repository work in mini-swe-agent, using pass@1 over scored attempts on its 113-task set. Provider, verifier and network errors are excluded by the publisher; context-window failures and agent timeouts count as failures. Its source publishes mean cost over scored attempts. Its publisher prices Astra's five costs at current API tariffs.

AA's Terminal-Bench 4.0 measures command-line work in AA's own scaffold. GDPval-AA v2.1 compares professional deliverables. AutomationBench-AA contributes the fraction of objectives completed, including partial credit; it is not the fraction of whole workflows completed. These are separate source runs from the AA Coding Agent Index.

Together, repository and terminal work carry 75% of Execute, while app workflows and documents retain 25%. Each component keeps its exact model, effort, benchmark version and harness. The mix is a workload proxy across those settings, not a measurement of one universal agent. Changing the provider filter changes access, never the benchmark harness or its score.

Fixed conversions preserve differences without depending on the current roster. Both mixes use:

Rate p                     → 100 × p
Published Omniscience O     → (O + 100) / 2
Native Briefcase/GDPval E   → (E − 500) / 20

Elo is not clipped: 1000 becomes 25 points, 1500 becomes 50 and 2500 becomes 100; higher or negative values remain possible. Twenty Elo per point is a declared cross-benchmark policy, not an empirically established exchange rate. 1000 Elo is not asserted to mean human-level performance. Measured scores do not change when another model is added or filtered. Estimates are re-fitted when the measurements change; UI filters do not re-fit them.

The November 11 capture uses GDPval-AA v2.1, Briefcase v1.1 and AA-Omniscience v1.1 (3,000 questions). Artificial Analysis Intelligence Index 5.0.1, which differs from 5.0 only in regrading CritPt v2, does not score or gate either mix; AA-LCR v1.1 and GDPval-AA v2.1 still do. AA re-fits both Elo scales within those versions; all current scores use one capture's results together. Differences from earlier captures are not evidence of model improvement. The weights and conversions above stay fixed.

Costs use the same weights as quality. AA bills use benchmark token counts, cache rates, prices and published fallback fractions divided by task count. When fallback pricing is unavailable, a separately marked estimate uses the primary model tariff; it is not a verified fallback-inclusive bill. Briefcase analytical quality uses the full Briefcase task bill. Direct DeepSWE costs are already per scored attempt. The resulting cost prices the declared benchmark mix, not the reader's next job or subscription quota.

Missing results and estimates

A missing measurement stays missing in the source. The estimator tries each missing component against available components in the exact configuration. Quality uses a straight line or a curve that levels off near the top, whichever predicts held-out models better for that pair of benchmarks; cost uses a straight line in log dollars. A predictor score beyond every trained configuration is read at the highest or lowest trained value. A predicted score never goes past the best or below the worst measured score on its benchmark. It never sits above the same model's measured higher effort, or below its measured lower effort, by more than the largest such inversion measured on that benchmark between reasoning efforts; a non-reasoning setting, which is not simply a lower effort, allows the larger of that and the largest inversion measured between non-reasoning and reasoning settings. A prediction held at any of those edges says so. Each training model receives equal total weight. Validation hides all efforts of a model together, and the predictor must beat a model-balanced benchmark-mean baseline across at least six other models. Predictions never train on other predictions or borrow an adjacent effort as a measurement. Sparse rows can show predictions, but a recommendation still needs at least half its score weight directly measured.

An Estimated label and hollow chart point identify a configuration with estimated quality or cost. The evidence disclosure shows the measured share, missing components and estimate basis. Statistical predictions include a predictor, held-out error and range. That range adds weighted component error allowances, each the model-balanced 95th-percentile held-out error among the third of measured configurations whose predictor scores are nearest, widened for predictors outside the training range. With each model left out in turn, those allowances covered 90% of held-out configurations on average across benchmark pairs: 89% in each board's bottom quarter and 91% in its top quarter. A single allowance for the whole board covered 84% in the bottom quarter. Coverage varies by pair: for DeepSWE predicted from Terminal-Bench 4.0 it is 85% overall, with a top-quarter allowance a third as wide as the whole-board one. It is an empirical missing-input range, not a confidence interval. Small measured-score differences also need not matter in practice.

Known token accounting with captured fallback tariffs but missing fractions supplies cost bounds from the cheaper and dearer rates in each token category. Their midpoint is a display convention, not an expected bill. If AA publishes a per-benchmark cost but no fallback tariff, its captured tokens priced at the primary tariff supply a scenario, with the actual fallback-inclusive bill and range unknown. Where a separate benchmark's cost is missing, a validated transfer may use that source-derived scenario as its predictor; the resulting estimate has no defensible bill range. Cost and quality estimates are separate; either may remain unavailable. The rail uses the displayed central estimates, so an estimated cost can exceed its indicative ceiling.

Fallback-enabled systems never inherit standalone or fallback-unknown measurements. A prediction about a system absent from the training overlap remains an explicit extrapolation. Generic model names do not establish dated releases. If the evidence is too sparse or a predictor fails validation, the row remains visible without a composite.

A measurement replaces an estimate when it arrives, with its error allowance, at the weekly refresh. An older benchmark version's result never fills a gap in the current one, and the weights stay fixed.

What this composition costs you

Different harnesses still matter. Direct DeepSWE and AA evaluate different scaffolds, tools and tasks. Owning the mix restores useful component coverage but cannot isolate model quality from those environments or predict performance inside Codex or Claude Code. The evidence table retains the source of each number so that trade-off stays inspectable.

Source quality is uneven. DeepSWE uses original engineering tasks and behavior-based verification, but covers a limited repository and language mix. AA's deliverable and analysis comparisons depend on model judges. AutomationBench's partial-credit objectives are easier to satisfy than complete workflows. FrontierCode, Zapier and HLE remain supporting evidence; they do not gate the mixes.

What the Figure lane's own sources are worth

The Figure mix emphasizes analysis, document reasoning and factual reliability. Its sources have their own limits:

  • HLE is supporting evidence, outside the mix. It has contested answers. FutureHouse examined 321 text-only chemistry and biology questions and found that 29% ± 3.7 had answers contradicted by peer-reviewed literature (FutureHouse, 23 July 2025). The HLE team's own follow-up on that page put roughly 18% of a reviewed subset as problematic under three-expert review and committed to rolling revisions. The finding covers that subset, not the whole exam.
  • AA-Omniscience's questions are machine-written. This capture uses AA-Omniscience v1.1, 3,000 questions. Artificial Analysis states the questions were generated by an LLM-based question-generation agent from authoritative academic and industry sources (AA, read 2026-10-28). The sources are human; the questions are not. An earlier public description named 6,000 questions; that count is not this version.
  • AA-LCR v1.1 corrected its own answers. AA's methodology records that v1.1 added a grading system prompt, corrected 16 answer keys and changed the grader, and that scores are not directly comparable with v1.0 (AA methodology, read 2026-09-08).
  • Briefcase is Artificial Analysis's own private benchmark. Its 91 tasks across four held-out scenarios were written by outside experts from firms including Google, McKinsey and BCG, but AA owns and runs it, only a fifth demonstration scenario is public, and AA's methodology table lists one repeat per task — no averaging over attempts (AA-Briefcase, read 2026-09-08). Its analytical-quality Elo is a preference ranking, which is not a correctness measure.

These are benchmark proxies, not success probabilities. Collaborative problem framing and planning remain unmeasured in Figure; selective escalation and supervision remain unmeasured in Execute. Agent harnesses, tools, task sets and judges differ. Shared AA provenance is not independent corroboration, and small numerical gaps need not matter in real work.

Data and limitscoverage, effort names, raw rows and sources

Which models are included

The comparison uses AA model evidence captured November 11, 2026, and direct DeepSWE v1.1, re-read that day; DeepSWE's source payload is unchanged since its September 22 generation. AA also prints a Daybreak Blue Sol max row with cybersecurity results only; it supplies no component of either mix and does not inherit the standard Sol measurements. The table includes the exact effort settings those sources publish for catalogued models, including rows that cannot form a full mix. It excludes rows AA estimates rather than measures, and the settings named below. The shortlist also preserves independent benchmark observations. Every result keeps its source effort, harness and system; measurements do not transfer across them.

An older model stays while something here depends on it: a frontier step, a route, a qualifying shortlist strength or value option, or a substitute for a named gap. Replacement within its vendor line starts that check; age alone never removes it. Opus 4.8 and Sonnet 5 left on September 29, replaced by Opus 5.5 and Sonnet 5.5. Mistral Large 4 Preview left on November 6, replaced by Mistral Large 4. GPT-5.4 Pro left on November 11, displaced by GPT-5.5 Pro; CritPt xhigh no longer qualifies as a shortlist strength after Fable 5.2 raised the ceiling.

Astra has no non-reasoning row in AA's capture, and current OpenAI documentation does not support that configuration. Muse Spark 1.3 max is publicly available and included; missing costs remain visible. Meta now documents native xhigh and Standard-tier Muse Spark 1.3 max; Contributor variants and Muse Code multi-agent runs do not inherit these measurements. The first-party DeepSeek Flash and Flash Vision aliases now serve V4.1 Flash, whose results cannot be inferred from the measured older versions; on October 17 DeepSeek upgraded V4.1 Flash to V4.1-Flash-1017, which AA prints as its own max, high and thinking-disabled rows, compared below as DeepSeek V4.1 Flash 1017, so the V4.1 Flash rows describe the September release rather than what the alias returns today. V4.1 Flash has its own measured max and thinking-disabled rows. GLM-5.3 is measured at low and max; its high setting is a named gap. AA prints partial results for Agnes 2.5 Pro Beta, with an estimated overall index and no Briefcase, AutomationBench or Terminal-Bench 4.0 result, and none for Agnes 3.0 Flash, which is not compared. Agnes 2.5 Pro Beta is not compared either: its AA-LCR, 83.0, is beaten by GPT-6 Luna max at 83.3, and its one shortlist observation is on a benchmark neither mix reads. Every measured non-reasoning row of a catalogued model is included like any other effort, including GPT-5.6 Luna, Sol and Terra, GPT-5.5, Qwen3.8 27B and DeepSeek V4 Pro 0813. SWE-2 has scoped FrontierCode evidence without a complete lane mix. Step 5 Preview keeps an undisclosed AA effort setting, so it is read as its vendor's default and labeled default (assumed). Qwen3.8 Max 0902 is stored at xhigh, which AA now names and Alibaba documents as the default. Qwen3.8-Flash-Next names no effort, so that row is read as the vendor default and labeled default (assumed). Step’s scoped task costs use AA’s captured $0.10 per million cached-input tokens; StepFun’s pricing page now lists the same rate; its October 8 version listed $0.05. Grok 4.7 low, high and xhigh have AA task costs alongside their measured Figure scores; xAI's reasoning guide also lists medium, which is unmeasured. MiMo-V2.6-Pro and MiMo-V2.6-Flash each have measured Figure evidence with thinking enabled, their native default. Grok 4.7 and both MiMo models lack a matching direct DeepSWE result; Execute uses separately labelled estimates where validation permits. Pro and Flash results do not transfer to each other, and UltraSpeed, thinking-disabled use and MiMo's open RL weights are unmeasured.

GPT-6 Sol and Luna each have six exact AA settings, with non-reasoning kept distinct from the five reasoning efforts. Neither has a matching direct DeepSWE result; the Execute lane labels any validated prediction separately. Opus 5.5 has five AA settings with its default safety fallback, and its Figure scores are measured. AA's fallback table lists a fallback tariff for each of the five efforts and, among the benchmarks used here, per-benchmark fractions for AA-Omniscience only. Its Omniscience cost is therefore a direct bill, and every other Opus 5.5 AA cost is a cost bound shown at its midpoint: the primary tariff is the low end and the fallback tariff the high end, so every Opus 5.5 Figure cost is a cost bound too. Opus 5.5 has no matching direct DeepSWE result: its Execute score predicts the missing 40%, and its direct DeepSWE task cost is predicted from its own AA-Omniscience bill, so its Execute costs carry a range. At max that bill is above every trained configuration, so the DeepSWE cost is held at the highest cost among the configurations it was trained on, $11.84, and the range is wide.

GPT-6.1 Sol has five exact AA settings, low to max, released September 29. OpenAI's model page lists those five efforts and excludes none and minimal; it mentions no fallback routing, so the rows are treated as standalone. It prices at $2 input, $0.10 cached input, $2.50 cache write and $10 output per million tokens. Its Figure scores are fully measured: max scores 72.2 at $0.76 a task, against Astra max at 71.4 and $3.19, which makes it the Codex Figure pick with no budget limit. The margin is 0.82 points, between two measured scores, so no estimate is involved; 0 of the 23 alternative weightings in the method sensitivity table put Astra max ahead. It has no matching direct DeepSWE result, and figures from other sources do not substitute for one, so its Execute score predicts the missing 40%: at max it reads about 63.8 (range 61.1 to 66.6) at about $1.50 (range $1.01 to $2.93), under the measured Astra xhigh at 65.9. GPT-6 Sol and GPT-5.6 Sol are superseded by it but stay, since a scoped shortlist strength or value option depends on each.

GPT-6.1 Terra has six exact AA settings, none through max, released October 26. OpenAI's model page lists those six efforts, with medium the API default, and excludes minimal; it mentions no fallback routing, so the rows are treated as standalone. It prices at $1 input, $0.05 cached input, $1.25 cache write and $5 output per million tokens. Figure is fully measured: max scores 65.6 at $0.29 a task, behind Sol 6.1 max at 72.2 and $0.76, so max is not a rung; medium is on Codex Figure, and low is too. Direct DeepSWE has no Terra 6.1 row, so Execute predicts the missing 40%: at max it reads about 62.3 (range 59.5 to 65.0) at about $1.25 (range $0.82 to $2.54). On All models Execute only low remains; medium and high sit on the Codex Execute frontier; max does not. GPT-5.6 Terra is superseded by it but stays, since a scoped shortlist strength or value option depends on it.

Grok 4.7 low scores 67.8 on Figure at $0.70 a task, against 70.3 at $1.99 for high. DeepSeek V4 Pro 0813's non-reasoning row, none in the chart, scores 36.0 at $0.15. AA now prices both none and max at DeepSeek's peak rates, $1.32 input and $3.96 output per million tokens. Off-peak rates are half those prices; the chart retains the captured peak basis.

Claude Haiku 5.5, released October 7, has five measured AA efforts, low to max; Anthropic's effort guide makes medium the API default. On October 8 AA removed Default Fallback from the same five row IDs, matching Anthropic's fallback guide: Haiku has no server-side fallback. Its Figure score at max is 68.1. AA restored lower-tier tariffs, but its row-specific caveat says the fivefold long-context pricing is not yet reflected. A separate release note says costs reflect tiered pricing; the captured payload still has only aggregate tokens and lower-tier tariffs, with no request-tier allocations.

Native bills stay missing, but the comparison includes a marked tier scenario. For Haiku max, the standard Figure mix gives $0.21–$1.05 per task, conditional on AA's captured tokens, cache-hit rate, five-minute cache writes and task counts. The rail uses the $0.63 midpoint scenario, not an expected price or an inferred premium share. The comparison shows whether the budget choice changes at the lower endpoint, midpoint and upper endpoint; these readings do not cover every possible request allocation. Other billing modifiers are outside the envelope. Anthropic's model page lists $0.10 input, $0.01 cached input and $0.50 output per million tokens through 100,000 prompt tokens, with fivefold rates above that threshold. AA also flags elevated pre-release AutomationBench refusals and plans reevaluation; the comparison retains the published scores. No effort has a matching direct DeepSWE measurement. Execute predicts that missing component using the existing model-held-out estimator, trained only on other models' measured results. Its cost is seeded from the marked AA tier scenario; the source accounting and transfer errors are not jointly bounded, so the total Execute bill has no established bound.

Mistral released Mistral Large 4 with its weights on October 29 (release post). The release post and the model card say mistral-large-4 now serves that build, which replaces the October 6 public preview; the preview's measurement does not transfer, and the API's model page still describes the preview. It takes low, medium or high, with high the recommended default (reasoning guide), and the release post lists $0.70 input, $0.07 cached input and $2.10 output per million tokens, which are AA's captured rates. AA measures high and medium; low is unmeasured. Neither measured setting has an AA-LCR result, so Figure predicts that share: high reads about 59.2 (range 57.4 to 61.0) at about $0.37 (range $0.34 to $0.55) a task, behind MiMo-V2.6-Pro at 64.3 and $0.13. Execute predicts the missing DeepSWE and GDPval-AA shares and reads about 49.3 (range 44.7 to 53.9) at about $2.86 (range $2.29 to $3.95), behind MiMo-V2.6-Flash at 49.9 and about $0.33, itself an estimate. Neither lane puts either setting on a frontier. The weights, and a separate NVFP4 checkpoint, are published under Mistral's own open-weight license; neither inherits the API's scores or costs.

Moonshot released Kimi K3.1 Pro on October 23 as kimi-k3.1-pro (quickstart). It always thinks, takes low, high or max with max the default, and lists $2 input, $0.20 cached input and $10 output per million tokens, which are AA's captured rates. Moonshot published the weights on November 2 under its own license; self-hosted costs are not priced here. AA records the effort it ran as max, which is Moonshot's default; low and high do not inherit it. Figure is fully measured at 74.0 and $2.25 a task, between Opus 5.5 medium at 73.6 and about $1.96 and high at 75.3 and about $2.76. Direct DeepSWE has no Kimi K3.1 Pro row, so Execute predicts the missing 40% and reads about 70.3 (range 67.4 to 73.3) at about $3.54 (range $2.95 to $4.63), above Sonnet 5.5 max at 68.8 and about $7.27; the two score ranges overlap. Its Terminal-Bench 4.0 score, 67.2%, is above every trained configuration, so the prediction is held at the edge of the measured range and flagged. On that estimate it leads All models Execute at about half Sonnet 5.5 max's cost. Kimi K3 stays: a scoped shortlist strength still depends on it.

MiniMax released MiniMax-M3.1-Flash on October 19 (launch post). It always thinks, takes low, medium, high, xhigh or max with max the default (API reference), and lists $0.15 input, $0.015 cached input and $0.60 output per million tokens, which are AA's captured rates. AA records the effort it ran as max, which is MiniMax's default; it is MiniMax's only model here. Figure is fully measured at 59.8 and $0.16 a task, behind MiMo-V2.6-Pro at 64.3 and $0.13. Direct DeepSWE has no row, so Execute predicts the missing 40% and reads about 51.9 (range 48.5 to 55.3) at about $0.87 (range $0.54 to $1.88), behind Sol 6.1 medium, which scores 59.6 at about $0.67. Its strongest result is AutomationBench-AA partial credit, 72.5 at $0.066 a task, which qualifies it as a scoped value option on the shortlist. MiniMax's own DeepSWE v1.1 figure of 73.7% is a vendor statement and is not scored; the direct board lists no MiniMax result. Self-hosted weights are not verified.

DeepSeek upgraded V4.1 Flash to V4.1-Flash-1017 on October 17; deepseek-flash serves it, and the retired deepseek-v4-flash names are still accepted and routed to it, at the Flash prices AA captured; AA's costs use the peak rates, and off-peak is half. Its weights are published under MIT, and a self-hosted copy inherits none of these results. AA measures max, high, the API default, and thinking disabled, none in the chart; low is unmeasured and the September release's results do not transfer to this build. At max Figure is fully measured at 64.0 and $0.21 a task. Execute predicts the missing DeepSWE share and reads about 55.9 (range 52.5 to 59.4) at about $0.86 (range $0.53 to $1.84). Its GDPval-AA result, 62.3 converted points at $0.35 a task, qualifies it as a scoped value option. high has no AA-LCR or GDPval-AA result: Figure reads about 58.5 (range 56.4 to 60.2) at about $0.13 (range $0.11 to $0.21) and Execute about 52.7 (range 48.1 to 57.3) at about $0.78 (range $0.46 to $1.71). Thinking disabled scores 52.8 on Figure, fully measured, at $0.25, and about 39.0 (range 29.1 to 48.9) on Execute at about $0.57 (range $0.31 to $1.33). DeepSeek's own DeepSWE v1.1 figure of 77.0 is a vendor statement and is not scored; the direct board lists no result for this build. No setting is on an All models frontier; with OpenAI models excluded, neither is. Figure max sits below MiMo-V2.6-Pro and the September V4.1 Flash max; Muse Ember takes the other Figure rows and the Execute steps.

Gemini 4 Argon reached general availability in the Gemini API on October 13 (changelog); Google announced it on September 30 for vetted cyber defenders only, and the model id is gemini-4-argon. Google's thinking guide lists low, medium and high, with medium the default. AA measures High only, on a row dated September 30, before general availability, and does not say how it ran the model, so the comparison assumes the row is the model Google now serves. Argon is compared at high only. medium, the default, is a named gap; FrontierCode and Zapier also print medium, supporting evidence that scores neither mix. Costs keep AA's captured rates of $4 input, $0.20 cached input and $20 output per million tokens; AA prints no cache-write tariff. Google's pricing page still lists $2 / $0.10 / $10 through November 30 and doubles all three from December 1; this capture already stores the doubled rates. Figure is fully measured at 68.9 and $3.65 a task; the November 6 capture records fewer of Argon's input tokens as cache hits than the October 28 one did, at unchanged rates, so its costs rose. Direct DeepSWE has no Argon row, so Execute predicts the missing 40% and reads about 66.2 (range 63.5 to 69.0) at about $7.38. That is a higher estimated score than Astra xhigh's measured 65.9, at an estimated $7.38 (range $5.94 to $12.06) above Astra's measured $4.34, and the score range includes Astra's figure. Neither is a rung on the All models Execute frontier, because Kimi K3.1 Pro max scores more for less; Astra xhigh stays the Codex Execute pick. With Anthropic and Moonshot models excluded, Carbon high leads All models Execute on its estimate and on measured components alone; Astra xhigh leads when every missing DeepSWE result is set to the measured average. Against Argon's estimate, Sonnet 5.5 max stays ahead only while its DeepSWE result is above 66.3. Google's own DeepSWE v1.1 figure of 77.9% is a vendor statement and is not scored; the direct board lists no Argon result.

Meta released Muse Wonder on October 12 as muse-wonder-1.0. Meta's model list calls it a preview in the Model API, at $4 input, $0.20 cached input and $20 output per million tokens. The reasoning page lists minimal through max, rejects none, and says max on this model does not select Muse Wonder Full. AA measures max and high. At max Figure is fully measured at 71.2 and $2.76 a task, behind Sol 6.1 max at 72.2 and $0.76; Execute predicts the missing DeepSWE share and reads about 63.6 (range 60.8 to 66.3) at about $6.74 (range $5.38 to $11.17). high has no AA-LCR or GDPval-AA result: Figure reads about 70.0 (range 68.0 to 71.2) at about $1.79 (range $1.73 to $2.09) and Execute about 57.7 (range 54.3 to 60.8) at about $4.74 (range $3.89 to $6.34). Neither setting is on an All models frontier; with OpenAI models excluded, high is a Figure rung. Muse Wonder Full, which AA also prints at Max, is for Muse Code Power and Maximum subscribers only, with no API and so no comparable task price. It is a named absence, not a row: its AA results do not transfer to Muse Wonder, and Meta's own DeepSWE claims for the Muse Wonder line, which cover the standard model as well, are not scored.

Thinking Machines released Inkling 2 Small on November 5 (release post). Tinker samples it as thinkingmachines/Inkling-2-Small, with a 64K context, and as a 256K variant, at $0.30 input, $0.03 cached input and $1.00 output per million tokens, which are AA's captured rates; serverless inference, in beta with a waitlist for production use, serves only a 256K NVFP4 build. AA measured a release candidate and does not say which endpoint, context or precision it called. The 64K id cannot hold AA-LCR's inputs, which need at least 128K, so the row did not come from it, and whether it describes the 256K variant or the NVFP4 build is not published. Effort is a number from 0 to just under 1, with named presets from none to xhigh and high, 0.9, the default (effort guide). AA does not say which effort it measured, so the row is read as high and labeled default (assumed). The release post says its own evaluations ran at xhigh, but it prints its GDPval-AA, AA-LCR, AA-Omniscience and EnterpriseOps-Gym results as AA's, so it does not say which effort AA ran either. Figure is fully measured at 65.2 and $0.32 a task, behind the fully measured GPT-6.1 Sol medium at 68.6 and $0.20; it is a Figure rung only with OpenAI and DeepSeek models both excluded. Direct DeepSWE has no row, so Execute predicts the missing 40% and reads about 42.0 (range 32.8 to 51.3) at about $1.76 (range $1.39 to $2.43), behind GPT-6.1 Terra low at 42.7 and about $0.22, itself an estimate. Its AnalystAgent and EnterpriseOps-Gym results are scoped shortlist strengths. Its own DeepSWE v1.1 figure of 72.1% is a vendor statement and is not scored. Weights are closed; the release post says they will follow under Apache 2.0.

Tencent made Hy4 generally available on November 4 as hy4 on TokenHub, its OpenAI-compatible API, at a list price of $0.834 input, $0.042 cached input and $2.501 output per million tokens, which are AA's captured rates. Until December 31, 2026, Tencent's billing terms settle Hy4 at half its list price, while TokenHub's page headlines 75% off without stating terms; the costs here stay at list. AA prints one Hy4 row, a pre-release build dated November 3 at hy4's list rate, which hy4-preview shares. Nothing published ties that build, or a serving precision, to either id, so the row is compared as a related variant: its results are not verified to transfer to hy4. Tencent's deep reasoning guide sets hy4 to high by default and also offers no_think; AA does not name the effort it ran, so the row is read as high and labeled default (assumed), and no_think is unmeasured. It has no AA-LCR result, so Figure predicts that share and reads about 60.8 (range 58.8 to 62.8) at about $0.44 (range $0.41 to $0.64) a task, behind GPT-6.1 Sol low at 63.9 and $0.11. GDPval-AA and direct DeepSWE are missing too, so Execute predicts half its score: about 56.2 (range 52.5 to 59.9) at about $1.35 (range $0.86 to $2.79), behind GPT-6.1 Sol medium at 59.6 and about $0.67, itself an estimate. Muse Ember's cheaper Execute rungs now sit below Sol 6.1 medium, so Hy4 is not an Execute rung even with OpenAI models excluded. Tencent has not published Hy4's weights.

Anthropic released Claude Fable 5.2 on November 9 as claude-fable-5-2. Adaptive thinking is always on; effort is low through max with high the API default (overview). It prices at $8 input, $0.20 cached input and $40 output per million tokens, which are AA's captured rates; the $16 / $80 fast setting is unmeasured, and Mythos 5.2 is a separate limited-availability id. AA measures all five efforts with Default Fallback. Figure max scores 81.1 at about $18.21 (range $18.13 to $18.28) a task, against Opus 5.5 max at 79.4 and about $9.27, which makes it the All models and Claude Code Figure pick with no budget limit. Direct DeepSWE has no Fable 5.2 row, so Execute predicts the missing 40%: at max it reads about 68.0 (range 65.3 to 70.8) at about $17.05 (range $14.24 to $26.01). medium is on All models Execute, between Sol 6.1 max and Kimi K3.1 Pro max. Claude Code requires 2.1.321 or newer. A launch offer includes Fable on Pro and Team standard seats through 2026-11-16 23:59 PT. Fable 5.1 stays: a scoped shortlist strength still depends on it.

Google made Gemini 4 Carbon generally available in the Gemini API on November 10 (changelog); the model id is gemini-4-carbon. Google's thinking guide lists low, medium and high, with medium the default. AA measures all three, on rows dated October 28. Costs keep AA's captured rates of $4 input, $0.20 cached input and $20 output per million tokens, the rate Google lists from December 1; Google's pricing page still lists $2 / $0.10 / $10 through November 30. Figure is fully measured at 72.1 and $2.73 a task at high. Direct DeepSWE has no Carbon row, so Execute predicts the missing 40% and reads about 69.7 (range 66.8 to 72.6) at about $5.90 (range $4.67 to $9.89). That is the All models Execute runner-up, at more than Kimi K3.1 Pro max costs for a lower score, so it is not a rung. medium has no AA-LCR result: Figure reads about 70.9 (range 68.9 to 71.5) at about $2.21 (range $2.14 to $2.56). low likewise: Figure reads about 68.3 (range 66.2 to 69.1) at about $2.24 (range $2.18 to $2.60). Neither setting is on an All models frontier.

Meta released Muse Ember on November 9 as muse-ember-1.0. Meta's docs list it in Standard only, at $0.10 input, $0.004 cached input and $0.50 output per million tokens up to 200K input tokens, and $0.30 / $0.012 / $1.50 for the whole request above that; AA's captured rates are the $0.10 / $0.50 tier. The reasoning page lists minimal through max, with max the default. AA measures max and xhigh. At max Figure reads about 63.9 (range 61.9 to 66.0) at about $0.050 (range $0.042 to $0.093) a task; Execute predicts half its score: about 58.4 (range 55.0 to 61.9) at about $0.36 (range $0.19 to $0.89). Both settings sit on the All models Figure and Execute frontiers, between Luna 6 xhigh and MiMo-V2.6-Pro on Figure and between Terra 6.1 low and Sol 6.1 medium on Execute. With OpenAI models excluded, Muse Ember max is an Execute rung, between Muse Ember xhigh and Fable 5.2 low. minimal, low, medium and high are unmeasured. Contributor is not available. Direct DeepSWE is unmeasured.

Ling 3.0 Flash VL was a scoped document-reasoning value option while AA priced it. Its author card publishes separate MIT weights; the vendor changelog dates public API access to September 4, while AA dates the row September 10. AA leaves effort undisclosed, so the row is read as the vendor default and labeled default (assumed). AA withdrew its price fields on October 14, so its native task costs are unavailable; the official price page is not substituted, and the measured scores stay without a cost. It has no matching direct DeepSWE result.

Ling 3.1 Flash, JT-4.1 Flash 236B A21B and Solar Mini 4 have AA rows but no result that qualifies for the shortlist, so none is catalogued. Ling 3.1 Flash 1019 (October 22) has every core Figure component measured but is not compared: on Figure it scores 64.1 at $1.18 a task, beaten on score and cost by MiMo-V2.6-Pro, 64.3 at $0.13 a task; on Execute, with DeepSWE predicted, it is beaten by Sol 6.1 medium, which scores 59.6 at about $0.67, and still is with DeepSWE set to the best measured 74.1. Nemotron 3.5 Bolt E4B has no published component scores. Nanbeige4.3-33B (October 25) publishes Apache 2.0 weights but has no API price or hosted endpoint, so it has no task cost and is not compared.

Sparse predictions appear for other models, but rows with less than half the score weight measured cannot become recommendations. GPT-5.5 Pro xhigh has no exact component evidence to seed a model-specific estimate. The shortlist states evidence limits beside each model.

The Execute leader and its limits

With no budget limit, as of the November 11 capture, Kimi K3.1 Pro max leads Execute for All models, at about 70.3 and about $3.54 (range $2.95 to $4.63) a task, and Sonnet 5.5 max leads for Claude Code, at about 68.8 and about $7.27. The Kimi K3.1 Pro lead rests on the predicted DeepSWE result. Its score range overlaps the range for Gemini 4 Carbon high, so the answer says close call. On the measured 60% of the score alone it still leads. Terminal-Bench 4.0 is 67.2% against 65.2% on Carbon and GDPval-AA 1861 against 1733 Elo; AutomationBench is 73.3% against 77.8% on Carbon. Neither has a matching direct DeepSWE result, and both are predicted 72.6 on it, just under the best measured 74.1. Both Terminal-Bench 4.0 scores are above every trained configuration, so each is read at the highest trained score, 59.6, and both predictions are flagged.

For Claude Code, Sonnet 5.5 max's range overlaps the range for Fable 5.2 max, so that lead is not settled either, and the answer says close call. On the measured 60% alone Sonnet 5.5 still leads. Terminal-Bench 4.0 is 63.6% against 59.6% on Fable 5.2 and GDPval-AA 1838 against 1831 Elo; AutomationBench is 71.8% against 76.4% on Fable 5.2. Fable 5.2 max also has no direct DeepSWE result and is predicted 72.6. Like every fallback-enabled system, each prediction is also an extrapolation from systems measured without a fallback. Sonnet 5.5 comes to about $7.27 per Execute task and Fable 5.2 to about $17.05 (range $14.24 to $26.01).

For All models the lead also rests on that prediction. The fully measured Astra xhigh scores 65.9; Kimi K3.1 Pro stays ahead of it only while its DeepSWE result is above 61.5. Of the configurations with a measured DeepSWE result, the five with the highest Terminal-Bench 4.0 scores, Astra at medium to max and Opus 5 max, reach 72.8 to 74.1, so the lead is plausible but not measured. The leaders were also scored four ways: as published, with a plain straight line and no bounds, with every missing DeepSWE result set to the measured average of 56.7, and on measured components alone. Only the measured average changes the All models leader, to Astra xhigh; the Claude Code leader is the same in all four. For Codex, Astra xhigh leads Execute with a margin of 0.2 points over Astra max. The range for GPT-6.1 Sol max includes Astra xhigh's score, and the plain straight line, which has no bounds, would put GPT-6.1 Sol max first at 67.3, so whether Astra xhigh stays ahead depends on the estimate for GPT-6.1 Sol's missing DeepSWE result.

Sonnet 5.5 has five AA settings with the default safety fallback. Anthropic's Sonnet 5.5 release notes say the fallback sends requests Sonnet 5.5 refuses on cybersecurity or AI-development grounds to Sonnet 5, whose input and output list prices match. Anthropic's platform release notes date Sonnet 5.5's cache-read reduction from $0.20 to $0.10 per million tokens to October 7; AA's capture uses the new rate. AA publishes no route or token split for those retries, so how many tokens the retries use, and whether AA's token counts include them, are unknown; every Sonnet 5.5 AA cost is therefore a primary-tariff scenario without a range. At max, Sonnet 5.5 comes to about $7.27 per Execute task and Opus 5.5 to about $12.65 (range $6.74 to $31.54). Both include an estimated cost for the missing DeepSWE share: Sonnet 5.5's is predicted from its own Terminal-Bench 4.0 cost, a primary-tariff scenario, and Opus 5.5's from its own AA-Omniscience bill, held at the edge of the measured range, with the rest of its AA costs at the midpoint of their cost bounds. The gap between them says little, and neither is a measured Execute bill.

Supporting benchmarks and missing results

Additional evidence can qualify a scoped shortlist alternative without changing the broad score. Figure includes GDP.pdf, CritPt, AnalystAgent and ITBench. Execute includes FrontierCode, Zapier, SciCode, EnterpriseOps-Gym and APEX-Agents. Supporting observations never replace a missing core component. Coverage varies across exact configurations; not measured is a missing result, not poor performance.

AA-Briefcase separately reports analytical quality and presentation. A low presentation score does not arithmetically lower its separate analytical-quality Elo. Its rubric, analytical Elo, and overall Elo answer different questions; none is silently substituted for another here.

Where the numbers come from

Artificial Analysis, direct DeepSWE, Cognition, and Zapier supply the measurements and accounting. Missing measurements or task accounting remain visible; a track can be selected only when its mix is measured or completed by a validated estimate.

Effort dials, model by model

  • GPT-6 Astra has measured low, medium, high, xhigh, and max configurations here. OpenAI's model guide excludes none; Pro is a separate class.
  • GPT-6.1 Sol has measured low, medium, high, xhigh, and max configurations here, with medium the API default. OpenAI's model guide excludes none and minimal.
  • GPT-6.1 Terra has measured none, low, medium, high, xhigh, and max configurations here, with medium the API default. OpenAI's model page excludes minimal.
  • Claude's effort documentation distinguishes model-specific effort settings. The measured Fable configurations include the source's fallback behavior; the direct bill accounts for those fractions.
  • Gemini Flash uses low, medium, and high thinking levels here. Each is independently measured. Gemini 4 Argon uses the same three levels, with medium the default; only high is measured, and medium is a named gap. Gemini 4 Carbon uses the same three; AA reports all three.
  • Claude Fable 5.2 has measured low through max here, with high the API default. The measured configurations include the source's fallback behavior.
  • Muse Wonder documents minimal through max and rejects none; AA reports max and high.
  • Muse Ember documents minimal through max, with max the default; AA reports max and xhigh.
  • Mistral Large 4 accepts low, medium and high, with high the recommended default; AA reports high and medium.
  • Inkling 2 Small takes an effort number with presets from none to xhigh, high the default. AA does not disclose its effort, so the one row is read as high, the default, and labeled default (assumed); the vendor's own evaluations ran at xhigh.
  • Hy4 defaults to high and also offers no_think. AA does not disclose its effort, so the one row is read as high and labeled default (assumed); no_think is unmeasured.
  • MiniMax M3.1 Flash always thinks and accepts low, medium, high, xhigh and max, with max the default. AA records the effort it ran as max.
  • Kimi K3.1 Pro always thinks and accepts low, high and max, with max the default. AA records the effort it ran as max.
  • DeepSeek V4.1 Flash exposes public API low, high (default), and max. medium and xhigh map to high; ultra maps to max; none disables thinking. Max and thinking disabled are measured here, on the September release; the alias has served V4.1-Flash-1017 since October 17, and that build has its own max, high and thinking-disabled rows.
  • Earlier DeepSeek versions and GLM document low / high / max; an unmeasured setting stays a named gap.
  • Qwen3.8-Flash-Next distinguishes its effort dial from disabling thinking. AA names no effort on the stored row, so it is read as the vendor default and labeled default (assumed).

What the prices do not include

Benchmark API cost is not your invoice. It includes the evaluated runs' recorded API usage, but does not price your extra retries, human review, latency, subscriptions, or the cost of a wrong answer. An API cost ratio is not a subscription-quota ratio. The rail prices a benchmark task mix as shares of a job; it does not forecast the cost of your next job, and the job split it uses is a control you set, not a measurement of you.

145 catalogued configurations · November 11, 2026 · ordered by Figure score. Each lane uses its declared job weights. ≈ marks an estimated score or cost; row details give its basis, any available range, measured gaps and recommendation eligibility. These scores are benchmark proxies, not probabilities of success.
Model · effortFigure$ / figure taskExecute$ / execute task
Claude Fable 5.2 · max
Evidence and eligibility
  • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
  • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
  • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
Figure estimate

100% of score weight measured. Score 81.1; no score component estimated. API cost ≈$18.21 per mix task; range $18.13–$18.28.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • AA-LCR 1.1 · cost: $1.05 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Execute estimate

60% of score weight measured. Score ≈68.0; missing-input range 65.3–70.8 points. API cost ≈$17.05 per mix task; range $14.24–$26.01.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • DeepSWE v1.1 · mini-swe-agent · cost: $6.28 per scored attempt, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GDPval-AA v2.1 · cost: $28.19 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

81.08≈$18.2067≈68.01≈$17.0538
Claude Opus 5.5 · max
Evidence and eligibility
  • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
  • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
  • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
  • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
  • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
  • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
Figure estimate

100% of score weight measured. Score 79.4; no score component estimated. API cost ≈$9.27 per mix task; range $6.59–$11.96.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • Briefcase analytical quality · cost: $29.90 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • AA-LCR 1.1 · cost: $0.76 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Execute estimate

60% of score weight measured. Score ≈67.2; missing-input range 64.4–69.9 points. API cost ≈$12.65 per mix task; range $6.74–$31.54.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • DeepSWE v1.1 · mini-swe-agent · cost: $11.84 per scored attempt, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: AA-Omniscience index. 8 training models, 17 configurations. Held-out mean absolute error 0.57 log dollars; benchmark-mean baseline 0.86.

    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the highest measured cost, so the prediction is held there; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Terminal-Bench 4.0 · AA harness · cost: $18.40 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GDPval-AA v2.1 · cost: $12.14 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • AutomationBench-AA partial credit · cost: $1.71 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

79.41≈$9.2736≈67.16≈$12.6463
Claude Fable 5.2 · xhigh
Evidence and eligibility
  • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
  • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
  • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
Figure estimate

100% of score weight measured. Score 79.2; no score component estimated. API cost ≈$8.87 per mix task; range $8.81–$8.94.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • AA-LCR 1.1 · cost: $0.97 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Execute estimate

60% of score weight measured. Score ≈68.0; missing-input range 65.2–70.8 points. API cost ≈$11.14 per mix task; range $9.08–$17.40.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • DeepSWE v1.1 · mini-swe-agent · cost: $6.28 per scored attempt, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GDPval-AA v2.1 · cost: $10.03 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

79.16≈$8.8735≈67.98≈$11.1389
Claude Fable 5.2 · high
Evidence and eligibility
  • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
  • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
  • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
Figure estimate

100% of score weight measured. Score 78.0; no score component estimated. API cost ≈$4.94 per mix task; range $4.87–$5.01.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • AA-LCR 1.1 · cost: $0.96 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Execute estimate

60% of score weight measured. Score ≈66.7; missing-input range 63.9–69.4 points. API cost ≈$7.49 per mix task; range $5.93–$12.27.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • DeepSWE v1.1 · mini-swe-agent · cost: $5.19 per scored attempt, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GDPval-AA v2.1 · cost: $4.41 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

77.98≈$4.9412≈66.69≈$7.4867
Claude Opus 5.5 · xhigh
Evidence and eligibility
  • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
  • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
  • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
  • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
  • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
  • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
Figure estimate

100% of score weight measured. Score 77.7; no score component estimated. API cost ≈$5.40 per mix task; range $3.87–$6.93.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • Briefcase analytical quality · cost: $17.30 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • AA-LCR 1.1 · cost: $0.67 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

Execute estimate

60% of score weight measured. Score ≈66.3; missing-input range 63.6–69.1 points. API cost ≈$6.41 per mix task; range $4.43–$8.84.

Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • DeepSWE v1.1 · mini-swe-agent · cost: $3.49 per scored attempt, estimated.

    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

    Predictor: AA-Omniscience index. 8 training models, 17 configurations. Held-out mean absolute error 0.57 log dollars; benchmark-mean baseline 0.86.

    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • Terminal-Bench 4.0 · AA harness · cost: $12.27 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • GDPval-AA v2.1 · cost: $5.62 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

  • AutomationBench-AA partial credit · cost: $1.07 per benchmark task, estimated.

    Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

77.74≈$5.3979≈66.31≈$6.4105
Claude Fable 5.1 · max
Evidence and eligibility
    • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
    Execute estimate

    60% of score weight measured. Score ≈62.2; missing-input range 59.5–65.0 points. API cost ≈$10.41 per mix task; range $8.70–$15.97.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • DeepSWE v1.1 · mini-swe-agent · cost: $6.19 per scored attempt, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    76.72$7.4199≈62.22≈$10.4120
    Claude Fable 5.2 · medium
    Evidence and eligibility
    • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
    • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
    • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
    Figure estimate

    100% of score weight measured. Score 76.0; no score component estimated. API cost ≈$2.34 per mix task; range $2.28–$2.41.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • AA-LCR 1.1 · cost: $0.95 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    Execute estimate

    60% of score weight measured. Score ≈63.9; missing-input range 61.2–66.7 points. API cost ≈$3.45 per mix task; range $2.83–$4.58.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • DeepSWE v1.1 · mini-swe-agent · score: 72.2 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • DeepSWE v1.1 · mini-swe-agent · cost: $3.16 per scored attempt, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • GDPval-AA v2.1 · cost: $1.49 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    75.99≈$2.3421≈63.94≈$3.4505
    Claude Opus 5.5 · high
    Evidence and eligibility
    • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
    • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
    • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
    • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
    • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
    • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
    Figure estimate

    100% of score weight measured. Score 75.3; no score component estimated. API cost ≈$2.76 per mix task; range $2.06–$3.46.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • Briefcase analytical quality · cost: $8.52 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AA-LCR 1.1 · cost: $0.67 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    Execute estimate

    60% of score weight measured. Score ≈64.3; missing-input range 61.5–67.0 points. API cost ≈$3.76 per mix task; range $2.55–$5.40.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • DeepSWE v1.1 · mini-swe-agent · score: 72.4 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • DeepSWE v1.1 · mini-swe-agent · cost: $2.39 per scored attempt, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: AA-Omniscience index. 8 training models, 17 configurations. Held-out mean absolute error 0.57 log dollars; benchmark-mean baseline 0.86.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • Terminal-Bench 4.0 · AA harness · cost: $7.09 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • GDPval-AA v2.1 · cost: $1.96 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • AutomationBench-AA partial credit · cost: $0.85 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    75.29≈$2.7588≈64.29≈$3.7587
    Claude Fable 5.2 · low
    Evidence and eligibility
    • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
    • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
    • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
    Figure estimate

    100% of score weight measured. Score 75.1; no score component estimated. API cost ≈$1.97 per mix task; range $1.91–$2.04.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • AA-LCR 1.1 · cost: $0.95 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    Execute estimate

    60% of score weight measured. Score ≈63.4; missing-input range 60.7–66.1 points. API cost ≈$2.96 per mix task; range $2.40–$3.98.

    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

    • DeepSWE v1.1 · mini-swe-agent · score: 72.2 converted points, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • DeepSWE v1.1 · mini-swe-agent · cost: $2.86 per scored attempt, estimated.

      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    • GDPval-AA v2.1 · cost: $1.07 per benchmark task, estimated.

      Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

    75.06≈$1.9749≈63.40≈$2.9609
    Claude Fable 5.1 · xhigh
    Evidence and eligibility
      • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
      Execute estimate

      60% of score weight measured. Score ≈63.0; missing-input range 60.3–65.8 points. API cost ≈$8.72 per mix task; range $7.17–$13.75.

      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

      • DeepSWE v1.1 · mini-swe-agent · score: 72.2 converted points, estimated.

        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

        Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

        Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • DeepSWE v1.1 · mini-swe-agent · cost: $5.60 per scored attempt, estimated.

        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

        Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

        Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      74.97$5.8269≈63.03≈$8.7224
      Claude Sonnet 5.5 · max
      Evidence and eligibility
      • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
      • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
      • Figure · AA-Omniscience index: Fallback token fractions are not published for this benchmark
      • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
      • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
      • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
      • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
      Figure estimate

      100% of score weight measured. Score 74.9; no score component estimated. API cost ≈$6.30 per mix task; fallback-inclusive bill and range unknown.

      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

      • Briefcase analytical quality · cost: $20.47 per benchmark task, estimated.

        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • AA-LCR 1.1 · cost: $0.34 per benchmark task, estimated.

        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • AA-Omniscience index · cost: $0.15 per benchmark task, estimated.

        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      Execute estimate

      60% of score weight measured. Score ≈68.8; missing-input range 65.9–71.6 points. API cost ≈$7.27 per mix task; range unknown, because part of the cost is an accounting scenario.

      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

      • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

        Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • DeepSWE v1.1 · mini-swe-agent · cost: $5.02 per scored attempt, estimated.

        Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

        Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

        Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • Terminal-Bench 4.0 · AA harness · cost: $12.65 per benchmark task, estimated.

        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • GDPval-AA v2.1 · cost: $6.84 per benchmark task, estimated.

        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      • AutomationBench-AA partial credit · cost: $1.03 per benchmark task, estimated.

        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

      74.89≈$6.3047≈68.76≈$7.2746
      Kimi K3.1 Pro · max
      Evidence and eligibility
        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
        Execute estimate

        60% of score weight measured. Score ≈70.3; missing-input range 67.4–73.3 points. API cost ≈$3.54 per mix task; range $2.95–$4.63.

        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

        • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

          The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        • DeepSWE v1.1 · mini-swe-agent · cost: $3.17 per scored attempt, estimated.

          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

          Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        74.01$2.2476≈70.34≈$3.5352
        Claude Opus 5.5 · medium
        Evidence and eligibility
        • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
        • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
        • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
        • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
        • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
        • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
        Figure estimate

        100% of score weight measured. Score 73.6; no score component estimated. API cost ≈$1.96 per mix task; range $1.50–$2.42.

        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

        • Briefcase analytical quality · cost: $5.86 per benchmark task, estimated.

          Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

          Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        • AA-LCR 1.1 · cost: $0.66 per benchmark task, estimated.

          Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

          Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        Execute estimate

        60% of score weight measured. Score ≈61.8; missing-input range 59.1–64.6 points. API cost ≈$3.05 per mix task; range $2.04–$4.45.

        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

        • DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated.

          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

          Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

          Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        • DeepSWE v1.1 · mini-swe-agent · cost: $2.16 per scored attempt, estimated.

          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

          Predictor: AA-Omniscience index. 8 training models, 17 configurations. Held-out mean absolute error 0.57 log dollars; benchmark-mean baseline 0.86.

          Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

          Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        • Terminal-Bench 4.0 · AA harness · cost: $5.63 per benchmark task, estimated.

          Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

          Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        • GDPval-AA v2.1 · cost: $1.05 per benchmark task, estimated.

          Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

          Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        • AutomationBench-AA partial credit · cost: $0.77 per benchmark task, estimated.

          Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

          Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

        73.62≈$1.9592≈61.84≈$3.0549
        Claude Fable 5.1 · high
        Evidence and eligibility
          • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
          Execute estimate

          60% of score weight measured. Score ≈61.0; missing-input range 58.3–63.8 points. API cost ≈$6.54 per mix task; range $5.22–$10.83.

          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

          • DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated.

            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

            Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

            Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

          • DeepSWE v1.1 · mini-swe-agent · cost: $4.78 per scored attempt, estimated.

            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

            Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

            Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

          72.96$3.8732≈61.02≈$6.5420
          Claude Opus 5 · max72.34$5.577061.24$12.3602
          GPT-6.1 Sol · max
          Evidence and eligibility
            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
            Execute estimate

            60% of score weight measured. Score ≈63.8; missing-input range 61.1–66.6 points. API cost ≈$1.50 per mix task; range $1.01–$2.93.

            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

            • DeepSWE v1.1 · mini-swe-agent · score: 72.3 converted points, estimated.

              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

            • DeepSWE v1.1 · mini-swe-agent · cost: $1.81 per scored attempt, estimated.

              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

            72.23$0.7609≈63.82≈$1.4955
            Gemini 4 Carbon · high
            Evidence and eligibility
              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
              Execute estimate

              60% of score weight measured. Score ≈69.7; missing-input range 66.8–72.6 points. API cost ≈$5.90 per mix task; range $4.67–$9.89.

              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

              • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

              • DeepSWE v1.1 · mini-swe-agent · cost: $4.45 per scored attempt, estimated.

                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

              72.08$2.7327≈69.67≈$5.8971
              Claude Fable 5.1 · medium
              Evidence and eligibility
                • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                Execute estimate

                60% of score weight measured. Score ≈57.7; missing-input range 55.0–60.5 points. API cost ≈$5.28 per mix task; range $4.12–$9.04.

                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                • DeepSWE v1.1 · mini-swe-agent · score: 71.3 converted points, estimated.

                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                  Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                  Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                • DeepSWE v1.1 · mini-swe-agent · cost: $4.19 per scored attempt, estimated.

                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                  Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                  Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                71.60$3.0230≈57.71≈$5.2810
                GPT-6 Astra · max71.41$3.189465.68$6.6658
                Muse Wonder · max
                Evidence and eligibility
                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                  Execute estimate

                  60% of score weight measured. Score ≈63.6; missing-input range 60.8–66.3 points. API cost ≈$6.74 per mix task; range $5.38–$11.17.

                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                  • DeepSWE v1.1 · mini-swe-agent · score: 72.0 converted points, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  • DeepSWE v1.1 · mini-swe-agent · cost: $4.93 per scored attempt, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  71.16$2.7552≈63.57≈$6.7417
                  Gemini 4 Carbon · medium
                  Evidence and eligibility
                  • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                  • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                  • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                  • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                  Figure estimate

                  70% of score weight measured. Score ≈70.9; missing-input range 68.9–71.5 points. API cost ≈$2.21 per mix task; range $2.14–$2.56.

                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                  • AA-LCR 1.1 · score: 81.9 converted points, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 42 training models, 122 configurations. Held-out mean absolute error 3.99 points; benchmark-mean baseline 4.35.

                    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  • AA-LCR 1.1 · cost: $0.27 per benchmark task, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Briefcase analytical quality. 36 training models, 100 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  Execute estimate

                  50% of score weight measured. Score ≈67.3; missing-input range 63.8–70.8 points. API cost ≈$5.27 per mix task; range $3.99–$9.23.

                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  • DeepSWE v1.1 · mini-swe-agent · cost: $4.15 per scored attempt, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  • GDPval-AA v2.1 · score: 53.9 converted points, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Briefcase analytical quality. 42 training models, 121 configurations. Held-out mean absolute error 3.45 points; benchmark-mean baseline 8.44.

                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  • GDPval-AA v2.1 · cost: $2.72 per benchmark task, estimated.

                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                    Predictor: Briefcase analytical quality. 36 training models, 98 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.26.

                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                  ≈70.95≈$2.2104≈67.28≈$5.2686
                  GPT-6 Astra · xhigh70.75$2.280765.93$4.3358
                  Grok 4.7 · xhigh
                  Evidence and eligibility
                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                    Execute estimate

                    60% of score weight measured. Score ≈52.1; missing-input range 48.7–55.5 points. API cost ≈$7.04 per mix task; range $5.66–$11.51.

                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                    • DeepSWE v1.1 · mini-swe-agent · score: 67.9 converted points, estimated.

                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • DeepSWE v1.1 · mini-swe-agent · cost: $4.98 per scored attempt, estimated.

                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    70.67$2.4925≈52.10≈$7.0408
                    Claude Sonnet 5.5 · xhigh
                    Evidence and eligibility
                    • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
                    • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
                    • Figure · AA-Omniscience index: Fallback token fractions are not published for this benchmark
                    • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                    • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
                    • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
                    • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
                    Figure estimate

                    100% of score weight measured. Score 70.6; no score component estimated. API cost ≈$2.19 per mix task; fallback-inclusive bill and range unknown.

                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                    • Briefcase analytical quality · cost: $6.98 per benchmark task, estimated.

                      Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • AA-LCR 1.1 · cost: $0.30 per benchmark task, estimated.

                      Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • AA-Omniscience index · cost: $0.0088 per benchmark task, estimated.

                      Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    Execute estimate

                    60% of score weight measured. Score ≈64.9; missing-input range 62.2–67.7 points. API cost ≈$3.75 per mix task; range unknown, because part of the cost is an accounting scenario.

                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                    • DeepSWE v1.1 · mini-swe-agent · score: 72.4 converted points, estimated.

                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • DeepSWE v1.1 · mini-swe-agent · cost: $3.41 per scored attempt, estimated.

                      Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                      Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                      Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • Terminal-Bench 4.0 · AA harness · cost: $6.08 per benchmark task, estimated.

                      Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • GDPval-AA v2.1 · cost: $1.95 per benchmark task, estimated.

                      Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    • AutomationBench-AA partial credit · cost: $0.42 per benchmark task, estimated.

                      Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                      Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                    70.62≈$2.1853≈64.94≈$3.7520
                    GPT-6 Astra · high70.50$1.728263.38$3.4234
                    Grok 4.7 · high
                    Evidence and eligibility
                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                      Execute estimate

                      60% of score weight measured. Score ≈51.3; missing-input range 47.9–54.7 points. API cost ≈$4.79 per mix task; range $4.06–$6.15.

                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                      • DeepSWE v1.1 · mini-swe-agent · score: 67.6 converted points, estimated.

                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                      • DeepSWE v1.1 · mini-swe-agent · cost: $3.85 per scored attempt, estimated.

                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                      70.31$1.9946≈51.33≈$4.7874
                      GPT-6.1 Sol · xhigh
                      Evidence and eligibility
                        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                        Execute estimate

                        60% of score weight measured. Score ≈63.0; missing-input range 60.3–65.8 points. API cost ≈$0.99 per mix task; range $0.63–$2.05.

                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                        • DeepSWE v1.1 · mini-swe-agent · score: 72.2 converted points, estimated.

                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                        • DeepSWE v1.1 · mini-swe-agent · cost: $1.34 per scored attempt, estimated.

                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                          Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                        70.24$0.3733≈63.04≈$0.9870
                        GPT-6.1 Sol · high
                        Evidence and eligibility
                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                          Execute estimate

                          60% of score weight measured. Score ≈61.6; missing-input range 58.8–64.3 points. API cost ≈$0.84 per mix task; range $0.53–$1.79.

                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                          • DeepSWE v1.1 · mini-swe-agent · score: 71.9 converted points, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          • DeepSWE v1.1 · mini-swe-agent · cost: $1.19 per scored attempt, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          70.18$0.3105≈61.58≈$0.8439
                          Muse Wonder · high
                          Evidence and eligibility
                          • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                          • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                          • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                          • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                          Figure estimate

                          70% of score weight measured. Score ≈70.0; missing-input range 68.0–71.2 points. API cost ≈$1.79 per mix task; range $1.73–$2.09.

                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                          • AA-LCR 1.1 · score: 81.2 converted points, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 42 training models, 122 configurations. Held-out mean absolute error 4.08 points; benchmark-mean baseline 4.33.

                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          • AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Briefcase analytical quality. 36 training models, 100 configurations. Held-out mean absolute error 0.87 log dollars; benchmark-mean baseline 1.13.

                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          Execute estimate

                          50% of score weight measured. Score ≈57.7; missing-input range 54.3–60.8 points. API cost ≈$4.74 per mix task; range $3.89–$6.34.

                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                          • DeepSWE v1.1 · mini-swe-agent · score: 70.4 converted points, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          • DeepSWE v1.1 · mini-swe-agent · cost: $3.97 per scored attempt, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          • GDPval-AA v2.1 · score: 60.4 converted points, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Briefcase analytical quality. 42 training models, 121 configurations. Held-out mean absolute error 3.57 points; benchmark-mean baseline 8.40.

                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          • GDPval-AA v2.1 · cost: $2.20 per benchmark task, estimated.

                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                            Predictor: Briefcase analytical quality. 36 training models, 98 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.26.

                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                          ≈70.01≈$1.7869≈57.72≈$4.7387
                          Claude Fable 5.1 · low
                          Evidence and eligibility
                            • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                            Execute estimate

                            60% of score weight measured. Score ≈55.1; missing-input range 52.4–57.9 points. API cost ≈$4.38 per mix task; range $3.68–$5.70.

                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                            • DeepSWE v1.1 · mini-swe-agent · score: 70.7 converted points, estimated.

                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                              Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                              Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                            • DeepSWE v1.1 · mini-swe-agent · cost: $3.73 per scored attempt, estimated.

                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                              Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                              Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                            69.49$2.4603≈55.14≈$4.3799
                            GPT-6 Astra · medium69.28$1.472361.12$3.1395
                            Kimi K3 · max
                            Evidence and eligibility
                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                              Figure estimate

                              60% of score weight measured. Score ≈69.2; missing-input range 60.6–78.3 points. API cost ≈$2.12 per mix task; range $2.11–$2.19.

                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                              • AA-Omniscience index · score: 63.3 converted points, estimated.

                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.21 points; benchmark-mean baseline 10.04.

                                The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                              • AA-Omniscience index · cost: $0.017 per benchmark task, estimated.

                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.37.

                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                              ≈69.17≈$2.119945.78$3.9523
                              Gemini 4 Argon · high
                              Evidence and eligibility
                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                Execute estimate

                                60% of score weight measured. Score ≈66.2; missing-input range 63.5–69.0 points. API cost ≈$7.38 per mix task; range $5.94–$12.06.

                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                • DeepSWE v1.1 · mini-swe-agent · score: 72.4 converted points, estimated.

                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                • DeepSWE v1.1 · mini-swe-agent · cost: $5.21 per scored attempt, estimated.

                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                68.88$3.6458≈66.23≈$7.3776
                                GPT-6.1 Sol · medium
                                Evidence and eligibility
                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                  Execute estimate

                                  60% of score weight measured. Score ≈59.6; missing-input range 56.9–62.4 points. API cost ≈$0.67 per mix task; range $0.40–$1.48.

                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                  • DeepSWE v1.1 · mini-swe-agent · score: 71.6 converted points, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • DeepSWE v1.1 · mini-swe-agent · cost: $1.02 per scored attempt, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  68.58$0.1972≈59.61≈$0.6750
                                  Gemini 4 Carbon · low
                                  Evidence and eligibility
                                  • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                  • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                  • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                  • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                  Figure estimate

                                  70% of score weight measured. Score ≈68.3; missing-input range 66.2–69.1 points. API cost ≈$2.24 per mix task; range $2.18–$2.60.

                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                  • AA-LCR 1.1 · score: 80.9 converted points, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 42 training models, 122 configurations. Held-out mean absolute error 3.99 points; benchmark-mean baseline 4.35.

                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • AA-LCR 1.1 · cost: $0.28 per benchmark task, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Briefcase analytical quality. 36 training models, 100 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  Execute estimate

                                  50% of score weight measured. Score ≈65.7; missing-input range 62.0–69.5 points. API cost ≈$5.05 per mix task; range $3.79–$8.96.

                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                  • DeepSWE v1.1 · mini-swe-agent · score: 72.6 converted points, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • DeepSWE v1.1 · mini-swe-agent · cost: $4.09 per scored attempt, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • GDPval-AA v2.1 · score: 52.5 converted points, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Briefcase analytical quality. 42 training models, 121 configurations. Held-out mean absolute error 3.45 points; benchmark-mean baseline 8.44.

                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • GDPval-AA v2.1 · cost: $2.77 per benchmark task, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Briefcase analytical quality. 36 training models, 98 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.26.

                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  ≈68.31≈$2.2432≈65.72≈$5.0461
                                  Claude Haiku 5.5 · max
                                  Evidence and eligibility
                                  • Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                  • Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                  • Figure · AA-Omniscience index: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                  • Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                  • Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                  • Execute · AutomationBench-AA partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                  Figure estimate

                                  100% of score weight measured. Score 68.1; no score component estimated. API cost ≈$0.63 per mix task; conditional tier envelope $0.21–$1.05.

                                  Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it.

                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                  • Briefcase analytical quality · cost: $2.02 per benchmark task, estimated.

                                    Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                    Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • AA-LCR 1.1 · cost: $0.058 per benchmark task, estimated.

                                    Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                    Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • AA-Omniscience index · cost: $0.018 per benchmark task, estimated.

                                    Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                    Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  Execute estimate

                                  60% of score weight measured. Score ≈50.3; missing-input range 46.8–53.7 points. API cost ≈$1.52 per mix task; range unknown, because part of the cost is an accounting scenario.

                                  Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound.

                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                  • DeepSWE v1.1 · mini-swe-agent · score: 69.5 converted points, estimated.

                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • DeepSWE v1.1 · mini-swe-agent · cost: $1.88 per scored attempt, estimated.

                                    Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • Terminal-Bench 4.0 · AA harness · cost: $1.96 per benchmark task, estimated.

                                    Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                    Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • GDPval-AA v2.1 · cost: $0.67 per benchmark task, estimated.

                                    Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                    Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  • AutomationBench-AA partial credit · cost: $0.13 per benchmark task, estimated.

                                    Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                    Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                  68.08≈$0.6320midpoint scenario · conditional $0.2107–$1.0533≈50.25≈$1.5212scenario + prediction · bill range unknown
                                  Claude Opus 5 · xhigh
                                  Evidence and eligibility
                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                    Figure estimate

                                    60% of score weight measured. Score ≈68.1; missing-input range 59.9–75.4 points. API cost ≈$4.52 per mix task; range $4.51–$4.65.

                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                    • AA-Omniscience index · score: 57.7 converted points, estimated.

                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                      Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.14 points; benchmark-mean baseline 10.01.

                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                    • AA-Omniscience index · cost: $0.035 per benchmark task, estimated.

                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                      Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.35.

                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                    ≈68.07≈$4.518259.49$10.3722
                                    Grok 4.7 · low
                                    Evidence and eligibility
                                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                      Execute estimate

                                      60% of score weight measured. Score ≈45.3; missing-input range 41.9–48.7 points. API cost ≈$3.18 per mix task; range $2.61–$4.25.

                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                      • DeepSWE v1.1 · mini-swe-agent · score: 64.3 converted points, estimated.

                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • DeepSWE v1.1 · mini-swe-agent · cost: $3.09 per scored attempt, estimated.

                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      67.79$0.6963≈45.33≈$3.1789
                                      Claude Sonnet 5.5 · high
                                      Evidence and eligibility
                                      • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
                                      • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
                                      • Figure · AA-Omniscience index: Fallback token fractions are not published for this benchmark
                                      • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                                      • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
                                      • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
                                      • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
                                      Figure estimate

                                      100% of score weight measured. Score 67.7; no score component estimated. API cost ≈$1.07 per mix task; fallback-inclusive bill and range unknown.

                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                      • Briefcase analytical quality · cost: $3.27 per benchmark task, estimated.

                                        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • AA-LCR 1.1 · cost: $0.29 per benchmark task, estimated.

                                        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • AA-Omniscience index · cost: $0.0057 per benchmark task, estimated.

                                        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      Execute estimate

                                      60% of score weight measured. Score ≈58.0; missing-input range 55.3–60.8 points. API cost ≈$1.68 per mix task; range unknown, because part of the cost is an accounting scenario.

                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                      • DeepSWE v1.1 · mini-swe-agent · score: 71.2 converted points, estimated.

                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                        Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                        Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • DeepSWE v1.1 · mini-swe-agent · cost: $1.99 per scored attempt, estimated.

                                        Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                        Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                        Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • Terminal-Bench 4.0 · AA harness · cost: $2.19 per benchmark task, estimated.

                                        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • GDPval-AA v2.1 · cost: $0.69 per benchmark task, estimated.

                                        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      • AutomationBench-AA partial credit · cost: $0.31 per benchmark task, estimated.

                                        Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                        Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                      67.74≈$1.0717≈58.04≈$1.6774
                                      Qwen3.8 Max 0902 · xhigh
                                      Evidence and eligibility
                                        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                        Execute estimate

                                        60% of score weight measured. Score ≈56.1; missing-input range 53.4–58.9 points. API cost ≈$9.52 per mix task; range $7.81–$15.06.

                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                        • DeepSWE v1.1 · mini-swe-agent · score: 70.5 converted points, estimated.

                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                        • DeepSWE v1.1 · mini-swe-agent · cost: $6.17 per scored attempt, estimated.

                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                          Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                        67.73$5.9450≈56.11≈$9.5186
                                        Muse Spark 1.3 · max
                                        Evidence and eligibility
                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                          Execute estimate

                                          60% of score weight measured. Score ≈54.1; missing-input range 50.7–57.5 points. API cost ≈$3.48 per mix task; range $2.87–$4.61.

                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                          • DeepSWE v1.1 · mini-swe-agent · score: 69.6 converted points, estimated.

                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                          • DeepSWE v1.1 · mini-swe-agent · cost: $3.26 per scored attempt, estimated.

                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                          67.37$1.0321≈54.09≈$3.4771
                                          Grok 4.6 · xhigh66.57$1.530748.53$4.8498
                                          DeepSeek V4.1 Flash · max
                                          Evidence and eligibility
                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                          Figure estimate

                                          60% of score weight measured. Score ≈66.1; missing-input range 56.9–75.2 points. API cost ≈$0.21 per mix task; range $0.21–$0.23.

                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                          • AA-Omniscience index · score: 60.3 converted points, estimated.

                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                            Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                          • AA-Omniscience index · cost: $0.0025 per benchmark task, estimated.

                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                            Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                          Execute estimate

                                          60% of score weight measured. Score ≈52.5; missing-input range 49.1–55.9 points. API cost ≈$1.19 per mix task; range $0.77–$2.44.

                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                          • DeepSWE v1.1 · mini-swe-agent · score: 68.2 converted points, estimated.

                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                          • DeepSWE v1.1 · mini-swe-agent · cost: $1.58 per scored attempt, estimated.

                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                          ≈66.06≈$0.2082≈52.48≈$1.1861
                                          GPT-6 Sol · max
                                          Evidence and eligibility
                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                            Execute estimate

                                            60% of score weight measured. Score ≈58.2; missing-input range 55.4–60.9 points. API cost ≈$2.68 per mix task; range $2.17–$3.63.

                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                            • DeepSWE v1.1 · mini-swe-agent · score: 71.2 converted points, estimated.

                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            • DeepSWE v1.1 · mini-swe-agent · cost: $2.74 per scored attempt, estimated.

                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            66.04$0.8562≈58.16≈$2.6783
                                            Claude Opus 5.5 · low
                                            Evidence and eligibility
                                            • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
                                            • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
                                            • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                                            • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
                                            • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
                                            • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
                                            Figure estimate

                                            100% of score weight measured. Score 66.0; no score component estimated. API cost ≈$0.63 per mix task; range $0.52–$0.73.

                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                            • Briefcase analytical quality · cost: $1.42 per benchmark task, estimated.

                                              Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

                                              Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            • AA-LCR 1.1 · cost: $0.66 per benchmark task, estimated.

                                              Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

                                              Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            Execute estimate

                                            60% of score weight measured. Score ≈50.3; missing-input range 46.9–53.7 points. API cost ≈$1.81 per mix task; range $1.05–$3.55.

                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                            • DeepSWE v1.1 · mini-swe-agent · score: 69.2 converted points, estimated.

                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                              Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                              Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            • DeepSWE v1.1 · mini-swe-agent · cost: $1.72 per scored attempt, estimated.

                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                              Predictor: AA-Omniscience index. 8 training models, 17 configurations. Held-out mean absolute error 0.57 log dollars; benchmark-mean baseline 0.86.

                                              Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                              Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            • Terminal-Bench 4.0 · AA harness · cost: $2.88 per benchmark task, estimated.

                                              Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

                                              Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            • GDPval-AA v2.1 · cost: $0.25 per benchmark task, estimated.

                                              Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

                                              Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            • AutomationBench-AA partial credit · cost: $0.59 per benchmark task, estimated.

                                              Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

                                              Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                            66.01≈$0.6253≈50.29≈$1.8122
                                            Claude Opus 5 · high
                                            Evidence and eligibility
                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                              Figure estimate

                                              60% of score weight measured. Score ≈65.9; missing-input range 56.1–73.6 points. API cost ≈$3.36 per mix task; range $3.35–$3.49.

                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                              • AA-Omniscience index · score: 56.9 converted points, estimated.

                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.14 points; benchmark-mean baseline 10.01.

                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              • AA-Omniscience index · cost: $0.035 per benchmark task, estimated.

                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.35.

                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              ≈65.94≈$3.360258.77$7.4928
                                              Claude Fable 5 · max
                                              Evidence and eligibility
                                              • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                              • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                              • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                                              • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
                                              Figure estimate

                                              60% of score weight measured. Score ≈65.7; missing-input range 56.5–74.9 points. API cost ≈$7.19 per mix task; range $7.16–$7.42.

                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                              • AA-Omniscience index · score: 59.3 converted points, estimated.

                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              • AA-Omniscience index · cost: $0.062 per benchmark task, estimated.

                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              Execute estimate

                                              60% of score weight measured. Score ≈56.9; missing-input range 55.6–59.7 points. API cost ≈$13.31 per mix task; range $8.26–$22.49.

                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                              • DeepSWE v1.1 · mini-swe-agent · score: 71.0 converted points, estimated.

                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                                Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              • DeepSWE v1.1 · mini-swe-agent · cost: $7.38 per scored attempt, estimated.

                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                Predictor: GDPval-AA v2.1. 13 training models, 40 configurations. Held-out mean absolute error 0.50 log dollars; benchmark-mean baseline 0.78.

                                                The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                                Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              • Terminal-Bench 4.0 · AA harness · cost: $26.18 per benchmark task, estimated.

                                                Midpoint of per-token-category primary/fallback allocation bounds; not an expected bill or confidence interval.

                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                              ≈65.72≈$7.1856≈56.94≈$13.3102
                                              GPT-6.1 Terra · max
                                              Evidence and eligibility
                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                Execute estimate

                                                60% of score weight measured. Score ≈62.3; missing-input range 59.5–65.0 points. API cost ≈$1.25 per mix task; range $0.82–$2.54.

                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                • DeepSWE v1.1 · mini-swe-agent · score: 72.3 converted points, estimated.

                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                • DeepSWE v1.1 · mini-swe-agent · cost: $1.62 per scored attempt, estimated.

                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                65.55$0.2901≈62.26≈$1.2475
                                                Muse Spark 1.3 · xhigh
                                                Evidence and eligibility
                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                  Execute estimate

                                                  60% of score weight measured. Score ≈45.8; missing-input range 42.4–49.2 points. API cost ≈$3.16 per mix task; range $2.59–$4.23.

                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                  • DeepSWE v1.1 · mini-swe-agent · score: 64.5 converted points, estimated.

                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $3.09 per scored attempt, estimated.

                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                  65.55$0.8674≈45.83≈$3.1631
                                                  GPT-6 Astra · low65.52$0.720754.87$1.6604
                                                  Step 5 Preview · default (assumed)
                                                  Evidence and eligibility
                                                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                    Execute estimate

                                                    60% of score weight measured. Score ≈52.6; missing-input range 49.2–56.0 points. API cost ≈$3.20 per mix task; range $2.62–$4.28.

                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                    • DeepSWE v1.1 · mini-swe-agent · score: 69.6 converted points, estimated.

                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                    • DeepSWE v1.1 · mini-swe-agent · cost: $3.14 per scored attempt, estimated.

                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                    65.39$0.7479≈52.61≈$3.1977
                                                    Inkling 2 Small · default (assumed)
                                                    Evidence and eligibility
                                                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                      Execute estimate

                                                      60% of score weight measured. Score ≈42.0; missing-input range 32.8–51.3 points. API cost ≈$1.76 per mix task; range $1.39–$2.43.

                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                      • DeepSWE v1.1 · mini-swe-agent · score: 62.3 converted points, estimated.

                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                      • DeepSWE v1.1 · mini-swe-agent · cost: $2.13 per scored attempt, estimated.

                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                      65.23$0.3181≈42.03≈$1.7632
                                                      Claude Opus 5 · medium
                                                      Evidence and eligibility
                                                      • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                      • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                        Figure estimate

                                                        60% of score weight measured. Score ≈65.0; missing-input range 55.6–71.9 points. API cost ≈$1.81 per mix task; range $1.80–$1.94.

                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                        • AA-Omniscience index · score: 58.7 converted points, estimated.

                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                          Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.14 points; benchmark-mean baseline 10.01.

                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                        • AA-Omniscience index · cost: $0.034 per benchmark task, estimated.

                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                          Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.35.

                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                        ≈64.96≈$1.809852.73$4.8090
                                                        GPT-5.6 Sol · max
                                                        Evidence and eligibility
                                                        • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                        • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                          Figure estimate

                                                          60% of score weight measured. Score ≈64.8; missing-input range 55.7–74.0 points. API cost ≈$1.33 per mix task; range $1.33–$1.41.

                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                          • AA-Omniscience index · score: 60.3 converted points, estimated.

                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                            Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • AA-Omniscience index · cost: $0.020 per benchmark task, estimated.

                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                            Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          ≈64.83≈$1.334657.63$6.6141
                                                          Claude Haiku 5.5 · xhigh
                                                          Evidence and eligibility
                                                          • Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                          • Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                          • Figure · AA-Omniscience index: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                          • Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                          • Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                          • Execute · AutomationBench-AA partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                          Figure estimate

                                                          100% of score weight measured. Score 64.5; no score component estimated. API cost ≈$0.34 per mix task; conditional tier envelope $0.11–$0.57.

                                                          Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it.

                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                          • Briefcase analytical quality · cost: $1.10 per benchmark task, estimated.

                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • AA-LCR 1.1 · cost: $0.046 per benchmark task, estimated.

                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • AA-Omniscience index · cost: $0.0021 per benchmark task, estimated.

                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          Execute estimate

                                                          60% of score weight measured. Score ≈48.3; missing-input range 44.9–51.7 points. API cost ≈$1.15 per mix task; range unknown, because part of the cost is an accounting scenario.

                                                          Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound.

                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                          • DeepSWE v1.1 · mini-swe-agent · score: 68.8 converted points, estimated.

                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $1.56 per scored attempt, estimated.

                                                            Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • Terminal-Bench 4.0 · AA harness · cost: $1.38 per benchmark task, estimated.

                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • GDPval-AA v2.1 · cost: $0.28 per benchmark task, estimated.

                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          • AutomationBench-AA partial credit · cost: $0.083 per benchmark task, estimated.

                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                          64.48≈$0.3438midpoint scenario · conditional $0.1146–$0.5731≈48.27≈$1.1485scenario + prediction · bill range unknown
                                                          Grok 4.6 · high
                                                          Evidence and eligibility
                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                            Figure estimate

                                                            60% of score weight measured. Score ≈64.4; missing-input range 56.4–69.1 points. API cost ≈$1.25 per mix task; range $1.25–$1.29.

                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                            • AA-Omniscience index · score: 57.9 converted points, estimated.

                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                              Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.37 points; benchmark-mean baseline 10.20.

                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                            • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                              Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.36.

                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                            ≈64.44≈$1.252949.13$4.0936
                                                            MiMo-V2.6-Pro · reasoning
                                                            Evidence and eligibility
                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                              Execute estimate

                                                              60% of score weight measured. Score ≈54.9; missing-input range 52.2–57.6 points. API cost ≈$0.52 per mix task; range $0.29–$1.19.

                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                              • DeepSWE v1.1 · mini-swe-agent · score: 69.9 converted points, estimated.

                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $0.85 per scored attempt, estimated.

                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                              64.33$0.1305≈54.90≈$0.5170
                                                              DeepSeek V4 Flash Vision · max
                                                              Evidence and eligibility
                                                              • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                              • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                              Figure estimate

                                                              60% of score weight measured. Score ≈64.1; missing-input range 54.6–73.6 points. API cost ≈$0.85 per mix task; range $0.85–$0.88.

                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                              • AA-Omniscience index · score: 58.7 converted points, estimated.

                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                              • AA-Omniscience index · cost: $0.0035 per benchmark task, estimated.

                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                              Execute estimate

                                                              60% of score weight measured. Score ≈41.3; missing-input range 32.0–50.5 points. API cost ≈$3.03 per mix task; range $2.47–$4.08.

                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                              • DeepSWE v1.1 · mini-swe-agent · score: 61.6 converted points, estimated.

                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $3.02 per scored attempt, estimated.

                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                              ≈64.10≈$0.8523≈41.26≈$3.0312
                                                              DeepSeek V4.1 Flash 1017 · max
                                                              Evidence and eligibility
                                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                Execute estimate

                                                                60% of score weight measured. Score ≈55.9; missing-input range 52.5–59.4 points. API cost ≈$0.86 per mix task; range $0.53–$1.84.

                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                • DeepSWE v1.1 · mini-swe-agent · score: 69.7 converted points, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                • DeepSWE v1.1 · mini-swe-agent · cost: $1.23 per scored attempt, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                63.97$0.2140≈55.95≈$0.8625
                                                                GLM-5.3 · max63.96$1.747457.37$4.7146
                                                                Muse Ember · max
                                                                Evidence and eligibility
                                                                • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                Figure estimate

                                                                70% of score weight measured. Score ≈63.9; missing-input range 61.9–66.0 points. API cost ≈$0.050 per mix task; range $0.042–$0.093.

                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                • AA-LCR 1.1 · score: 81.2 converted points, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 43 training models, 123 configurations. Held-out mean absolute error 4.00 points; benchmark-mean baseline 4.25.

                                                                  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                • AA-LCR 1.1 · cost: $0.029 per benchmark task, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Briefcase analytical quality. 37 training models, 101 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                                                                  Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                Execute estimate

                                                                50% of score weight measured. Score ≈58.4; missing-input range 55.0–61.9 points. API cost ≈$0.36 per mix task; range $0.19–$0.89.

                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                • DeepSWE v1.1 · mini-swe-agent · score: 70.8 converted points, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                • DeepSWE v1.1 · mini-swe-agent · cost: $0.66 per scored attempt, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                • GDPval-AA v2.1 · score: 55.9 converted points, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Briefcase analytical quality. 43 training models, 122 configurations. Held-out mean absolute error 3.51 points; benchmark-mean baseline 8.48.

                                                                  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                • GDPval-AA v2.1 · cost: $0.071 per benchmark task, estimated.

                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                  Predictor: Briefcase analytical quality. 37 training models, 99 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.27.

                                                                  Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                ≈63.95≈$0.0497≈58.43≈$0.3650
                                                                GPT-6.1 Sol · low
                                                                Evidence and eligibility
                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                  Execute estimate

                                                                  60% of score weight measured. Score ≈50.4; missing-input range 47.0–53.8 points. API cost ≈$0.49 per mix task; range $0.27–$1.12.

                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 69.1 converted points, estimated.

                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.79 per scored attempt, estimated.

                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  63.90$0.1107≈50.43≈$0.4863
                                                                  Claude Sonnet 5.5 · medium
                                                                  Evidence and eligibility
                                                                  • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
                                                                  • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
                                                                  • Figure · AA-Omniscience index: Fallback token fractions are not published for this benchmark
                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                                                                  • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
                                                                  • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
                                                                  • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
                                                                  Figure estimate

                                                                  100% of score weight measured. Score 63.8; no score component estimated. API cost ≈$0.51 per mix task; fallback-inclusive bill and range unknown.

                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                  • Briefcase analytical quality · cost: $1.39 per benchmark task, estimated.

                                                                    Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • AA-LCR 1.1 · cost: $0.29 per benchmark task, estimated.

                                                                    Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • AA-Omniscience index · cost: $0.0031 per benchmark task, estimated.

                                                                    Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  Execute estimate

                                                                  60% of score weight measured. Score ≈50.4; missing-input range 47.0–53.8 points. API cost ≈$1.29 per mix task; range unknown, because part of the cost is an accounting scenario.

                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 68.9 converted points, estimated.

                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                                                    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $1.68 per scored attempt, estimated.

                                                                    Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                    Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                                                    Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • Terminal-Bench 4.0 · AA harness · cost: $1.58 per benchmark task, estimated.

                                                                    Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • GDPval-AA v2.1 · cost: $0.32 per benchmark task, estimated.

                                                                    Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  • AutomationBench-AA partial credit · cost: $0.26 per benchmark task, estimated.

                                                                    Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                    Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                  63.76≈$0.5064≈50.37≈$1.2942
                                                                  Grok 4.6 · medium
                                                                  Evidence and eligibility
                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                    Figure estimate

                                                                    60% of score weight measured. Score ≈63.7; missing-input range 53.9–68.2 points. API cost ≈$0.89 per mix task; range $0.89–$0.93.

                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                    • AA-Omniscience index · score: 58.3 converted points, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.37 points; benchmark-mean baseline 10.20.

                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.36.

                                                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    ≈63.65≈$0.890746.66$3.3272
                                                                    Muse Ember · xhigh
                                                                    Evidence and eligibility
                                                                    • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                    • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                    • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                    • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                    Figure estimate

                                                                    70% of score weight measured. Score ≈63.5; missing-input range 61.5–65.6 points. API cost ≈$0.038 per mix task; range $0.032–$0.081.

                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                    • AA-LCR 1.1 · score: 81.1 converted points, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 43 training models, 123 configurations. Held-out mean absolute error 4.00 points; benchmark-mean baseline 4.25.

                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    • AA-LCR 1.1 · cost: $0.025 per benchmark task, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: Briefcase analytical quality. 37 training models, 101 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                                                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    Execute estimate

                                                                    50% of score weight measured. Score ≈55.9; missing-input range 52.4–59.3 points. API cost ≈$0.32 per mix task; range $0.16–$0.80.

                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                    • DeepSWE v1.1 · mini-swe-agent · score: 70.0 converted points, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    • DeepSWE v1.1 · mini-swe-agent · cost: $0.60 per scored attempt, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    • GDPval-AA v2.1 · score: 55.2 converted points, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: Briefcase analytical quality. 43 training models, 122 configurations. Held-out mean absolute error 3.51 points; benchmark-mean baseline 8.48.

                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    • GDPval-AA v2.1 · cost: $0.055 per benchmark task, estimated.

                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                      Predictor: Briefcase analytical quality. 37 training models, 99 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.27.

                                                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                    ≈63.55≈$0.0382≈55.88≈$0.3240
                                                                    GPT-6.1 Terra · xhigh
                                                                    Evidence and eligibility
                                                                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                      Execute estimate

                                                                      60% of score weight measured. Score ≈58.4; missing-input range 55.7–61.1 points. API cost ≈$0.70 per mix task; range $0.42–$1.54.

                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                      • DeepSWE v1.1 · mini-swe-agent · score: 71.4 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • DeepSWE v1.1 · mini-swe-agent · cost: $1.06 per scored attempt, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      63.55$0.1541≈58.40≈$0.6973
                                                                      Claude Fable 5 · xhigh
                                                                      Evidence and eligibility

                                                                      Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                      Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                      • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                      • Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
                                                                      • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                      • Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
                                                                      • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                      • Figure · AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
                                                                      • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                      • Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration
                                                                      • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                      • Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration
                                                                      • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                      • Execute · AutomationBench-AA partial credit: Artificial Analysis has not published a model row for this configuration
                                                                      Figure estimate

                                                                      0% of score weight measured. Score ≈63.1; missing-input range 47.4–73.7 points. API cost ≈$4.64 per mix task; range $1.52–$14.16.

                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                      • Briefcase analytical quality · score: 49.5 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • Briefcase analytical quality · cost: $14.78 per benchmark task, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                        Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • AA-LCR 1.1 · score: 82.2 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • AA-LCR 1.1 · cost: $0.63 per benchmark task, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

                                                                        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                        Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • AA-Omniscience index · score: 59.0 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • AA-Omniscience index · cost: $0.041 per benchmark task, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                        Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      Execute estimate

                                                                      40% of score weight measured. Score ≈51.9; missing-input range 42.8–58.2 points. API cost ≈$13.03 per mix task; range $8.20–$26.17.

                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                      • Terminal-Bench 4.0 · AA harness · score: 29.3 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45.

                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • Terminal-Bench 4.0 · AA harness · cost: $19.29 per benchmark task, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the highest measured cost, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                        Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • GDPval-AA v2.1 · score: 52.4 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 43 configurations. Held-out mean absolute error 4.48 points; benchmark-mean baseline 7.08.

                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • GDPval-AA v2.1 · cost: $6.33 per benchmark task, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                        Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • AutomationBench-AA partial credit · score: 55.9 converted points, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18.

                                                                        A higher or lower effort of the same model was measured, so the prediction is held at Fable 5 max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (1.9 points). The held-out test does not establish accuracy here.

                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      • AutomationBench-AA partial credit · cost: $1.85 per benchmark task, estimated.

                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                        The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the highest measured cost, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                        Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                      ≈63.11≈$4.6377≈51.85≈$13.0269
                                                                      GPT-5.6 Sol · xhigh
                                                                      Evidence and eligibility
                                                                      • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                      • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                        Figure estimate

                                                                        60% of score weight measured. Score ≈62.9; missing-input range 53.7–72.1 points. API cost ≈$1.05 per mix task; range $1.04–$1.12.

                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                        • AA-Omniscience index · score: 59.3 converted points, estimated.

                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                          Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                        • AA-Omniscience index · cost: $0.020 per benchmark task, estimated.

                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                          Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                        ≈62.91≈$1.048350.64$3.3859
                                                                        GPT-6 Sol · xhigh
                                                                        Evidence and eligibility
                                                                        • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                        • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                        Figure estimate

                                                                        60% of score weight measured. Score ≈62.3; missing-input range 52.8–66.0 points. API cost ≈$0.40 per mix task; range $0.40–$0.44.

                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                        • AA-Omniscience index · score: 58.6 converted points, estimated.

                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                          Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.47 points; benchmark-mean baseline 10.25.

                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                        • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                          Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.02 log dollars; benchmark-mean baseline 1.35.

                                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                        Execute estimate

                                                                        60% of score weight measured. Score ≈52.3; missing-input range 48.9–55.7 points. API cost ≈$1.53 per mix task; range $1.03–$3.00.

                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                        • DeepSWE v1.1 · mini-swe-agent · score: 69.0 converted points, estimated.

                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                        • DeepSWE v1.1 · mini-swe-agent · cost: $1.85 per scored attempt, estimated.

                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                          Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                        ≈62.33≈$0.4016≈52.30≈$1.5256
                                                                        GPT-6.1 Terra · high
                                                                        Evidence and eligibility
                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                          Execute estimate

                                                                          60% of score weight measured. Score ≈56.1; missing-input range 53.3–58.8 points. API cost ≈$0.53 per mix task; range $0.30–$1.20.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 70.9 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $0.85 per scored attempt, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          62.21$0.1126≈56.07≈$0.5264
                                                                          Claude Fable 5 · high
                                                                          Evidence and eligibility

                                                                          Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                          Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                          • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                          • Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
                                                                          • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                          • Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
                                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                          • Figure · AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
                                                                          • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                          • Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration
                                                                          • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                          • Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration
                                                                          • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                          • Execute · AutomationBench-AA partial credit: Artificial Analysis has not published a model row for this configuration
                                                                          Figure estimate

                                                                          0% of score weight measured. Score ≈62.1; missing-input range 46.4–73.1 points. API cost ≈$3.45 per mix task; range $1.31–$9.12.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • Briefcase analytical quality · score: 48.4 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • Briefcase analytical quality · cost: $10.95 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                            Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-LCR 1.1 · score: 82.0 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-LCR 1.1 · cost: $0.51 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

                                                                            Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-Omniscience index · score: 57.5 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-Omniscience index · cost: $0.033 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                            Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          Execute estimate

                                                                          40% of score weight measured. Score ≈50.6; missing-input range 42.5–56.7 points. API cost ≈$10.75 per mix task; range $6.74–$20.08.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • Terminal-Bench 4.0 · AA harness · score: 27.5 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • Terminal-Bench 4.0 · AA harness · cost: $18.14 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                            Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • GDPval-AA v2.1 · score: 52.0 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 43 configurations. Held-out mean absolute error 4.48 points; benchmark-mean baseline 7.08.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • GDPval-AA v2.1 · cost: $4.70 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                            Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AutomationBench-AA partial credit · score: 55.9 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18.

                                                                            A higher or lower effort of the same model was measured, so the prediction is held at Fable 5 max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (1.9 points). The held-out test does not establish accuracy here.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AutomationBench-AA partial credit · cost: $1.74 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                            Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          ≈62.14≈$3.4525≈50.64≈$10.7514
                                                                          Claude Haiku 5.5 · high
                                                                          Evidence and eligibility
                                                                          • Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                          • Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                          • Figure · AA-Omniscience index: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                          • Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                          • Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                          • Execute · AutomationBench-AA partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                          Figure estimate

                                                                          100% of score weight measured. Score 62.1; no score component estimated. API cost ≈$0.21 per mix task; conditional tier envelope $0.072–$0.36.

                                                                          Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • Briefcase analytical quality · cost: $0.67 per benchmark task, estimated.

                                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-LCR 1.1 · cost: $0.045 per benchmark task, estimated.

                                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-Omniscience index · cost: $0.0012 per benchmark task, estimated.

                                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          Execute estimate

                                                                          60% of score weight measured. Score ≈44.0; missing-input range 40.5–47.4 points. API cost ≈$0.90 per mix task; range unknown, because part of the cost is an accounting scenario.

                                                                          Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 66.7 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $1.31 per scored attempt, estimated.

                                                                            Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • Terminal-Bench 4.0 · AA harness · cost: $0.99 per benchmark task, estimated.

                                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • GDPval-AA v2.1 · cost: $0.13 per benchmark task, estimated.

                                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AutomationBench-AA partial credit · cost: $0.068 per benchmark task, estimated.

                                                                            Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                            Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          62.08≈$0.2146midpoint scenario · conditional $0.0715–$0.3577≈43.95≈$0.8971scenario + prediction · bill range unknown
                                                                          GPT-6 Sol · high
                                                                          Evidence and eligibility
                                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                          Figure estimate

                                                                          60% of score weight measured. Score ≈62.0; missing-input range 52.8–65.1 points. API cost ≈$0.25 per mix task; range $0.25–$0.29.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • AA-Omniscience index · score: 60.1 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.47 points; benchmark-mean baseline 10.25.

                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.02 log dollars; benchmark-mean baseline 1.35.

                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          Execute estimate

                                                                          60% of score weight measured. Score ≈50.0; missing-input range 46.5–53.4 points. API cost ≈$1.32 per mix task; range $0.87–$2.66.

                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 68.1 converted points, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $1.69 per scored attempt, estimated.

                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                          ≈61.97≈$0.2534≈49.95≈$1.3190
                                                                          Qwen3.8-Flash-Next · default (assumed)
                                                                          Evidence and eligibility
                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                            Execute estimate

                                                                            60% of score weight measured. Score ≈50.0; missing-input range 46.6–53.4 points. API cost ≈$0.83 per mix task; range $0.51–$1.81.

                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 67.8 converted points, estimated.

                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $1.23 per scored attempt, estimated.

                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                            61.81$0.4875≈50.02≈$0.8334
                                                                            GPT-5.6 Sol · high
                                                                            Evidence and eligibility
                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                              Figure estimate

                                                                              60% of score weight measured. Score ≈61.1; missing-input range 51.9–70.3 points. API cost ≈$0.74 per mix task; range $0.74–$0.82.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • AA-Omniscience index · score: 58.9 converted points, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AA-Omniscience index · cost: $0.020 per benchmark task, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              ≈61.12≈$0.742648.36$2.3884
                                                                              GLM-5.3-Flash · max60.88$0.300951.65$0.5424
                                                                              Claude Sonnet 5.5 · low
                                                                              Evidence and eligibility
                                                                              • Figure · Briefcase analytical quality: Fallback token fractions are not published for this benchmark
                                                                              • Figure · AA-LCR 1.1: Fallback token fractions are not published for this benchmark
                                                                              • Figure · AA-Omniscience index: Fallback token fractions are not published for this benchmark
                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: AA fallback-enabled system has no matching direct DeepSWE measurement
                                                                              • Execute · Terminal-Bench 4.0 · AA harness: Fallback token fractions are not published for this benchmark
                                                                              • Execute · GDPval-AA v2.1: Fallback token fractions are not published for this benchmark
                                                                              • Execute · AutomationBench-AA partial credit: Fallback token fractions are not published for this benchmark
                                                                              Figure estimate

                                                                              100% of score weight measured. Score 60.9; no score component estimated. API cost ≈$0.33 per mix task; fallback-inclusive bill and range unknown.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • Briefcase analytical quality · cost: $0.80 per benchmark task, estimated.

                                                                                Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AA-LCR 1.1 · cost: $0.29 per benchmark task, estimated.

                                                                                Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AA-Omniscience index · cost: $0.0029 per benchmark task, estimated.

                                                                                Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              Execute estimate

                                                                              60% of score weight measured. Score ≈44.6; missing-input range 41.2–48.0 points. API cost ≈$1.11 per mix task; range unknown, because part of the cost is an accounting scenario.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 66.3 converted points, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                                                                Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $1.50 per scored attempt, estimated.

                                                                                Predicted from the named predictor benchmark’s primary-tariff scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                Fallback-enabled system absent from measured DeepSWE training overlap. The held-out test does not establish accuracy here.

                                                                                Missing measurement: AA fallback-enabled system has no matching direct DeepSWE measurement. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • Terminal-Bench 4.0 · AA harness · cost: $1.28 per benchmark task, estimated.

                                                                                Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • GDPval-AA v2.1 · cost: $0.21 per benchmark task, estimated.

                                                                                Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AutomationBench-AA partial credit · cost: $0.24 per benchmark task, estimated.

                                                                                Artificial Analysis benchmark tokens priced at the primary model tariff. AA publishes no fallback route or billed attempts for this system; the actual fallback-inclusive bill and its range are unknown.

                                                                                Missing measurement: Fallback token fractions are not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              60.87≈$0.3293≈44.59≈$1.1084
                                                                              Hy4 · default (assumed)
                                                                              Evidence and eligibility
                                                                              • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                              • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                              • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                              • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                              Figure estimate

                                                                              70% of score weight measured. Score ≈60.8; missing-input range 58.8–62.8 points. API cost ≈$0.44 per mix task; range $0.41–$0.64.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • AA-LCR 1.1 · score: 81.0 converted points, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 43 training models, 123 configurations. Held-out mean absolute error 4.00 points; benchmark-mean baseline 4.25.

                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AA-LCR 1.1 · cost: $0.11 per benchmark task, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Briefcase analytical quality. 37 training models, 101 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              Execute estimate

                                                                              50% of score weight measured. Score ≈56.2; missing-input range 52.5–59.9 points. API cost ≈$1.35 per mix task; range $0.86–$2.79.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 70.3 converted points, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $1.71 per scored attempt, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • GDPval-AA v2.1 · score: 51.4 converted points, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Briefcase analytical quality. 43 training models, 122 configurations. Held-out mean absolute error 3.51 points; benchmark-mean baseline 8.48.

                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • GDPval-AA v2.1 · cost: $0.58 per benchmark task, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Briefcase analytical quality. 37 training models, 99 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.27.

                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              ≈60.82≈$0.4418≈56.17≈$1.3524
                                                                              Claude Haiku 5.5 · medium
                                                                              Evidence and eligibility
                                                                              • Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                              • Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                              • Figure · AA-Omniscience index: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                              • Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                              • Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                              • Execute · AutomationBench-AA partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                              Figure estimate

                                                                              100% of score weight measured. Score 60.3; no score component estimated. API cost ≈$0.13 per mix task; conditional tier envelope $0.044–$0.22.

                                                                              Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • Briefcase analytical quality · cost: $0.40 per benchmark task, estimated.

                                                                                Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AA-LCR 1.1 · cost: $0.045 per benchmark task, estimated.

                                                                                Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AA-Omniscience index · cost: $0.00094 per benchmark task, estimated.

                                                                                Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              Execute estimate

                                                                              60% of score weight measured. Score ≈39.0; missing-input range 35.6–42.4 points. API cost ≈$0.61 per mix task; range unknown, because part of the cost is an accounting scenario.

                                                                              Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound.

                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 63.7 converted points, estimated.

                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $0.99 per scored attempt, estimated.

                                                                                Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • Terminal-Bench 4.0 · AA harness · cost: $0.58 per benchmark task, estimated.

                                                                                Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • GDPval-AA v2.1 · cost: $0.062 per benchmark task, estimated.

                                                                                Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              • AutomationBench-AA partial credit · cost: $0.057 per benchmark task, estimated.

                                                                                Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                              60.33≈$0.1334midpoint scenario · conditional $0.0445–$0.2224≈38.98≈$0.6144scenario + prediction · bill range unknown
                                                                              GPT-6.1 Terra · medium
                                                                              Evidence and eligibility
                                                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                Execute estimate

                                                                                60% of score weight measured. Score ≈51.9; missing-input range 48.5–55.3 points. API cost ≈$0.39 per mix task; range $0.21–$0.92.

                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                • DeepSWE v1.1 · mini-swe-agent · score: 69.4 converted points, estimated.

                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                • DeepSWE v1.1 · mini-swe-agent · cost: $0.67 per scored attempt, estimated.

                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                60.32$0.0843≈51.87≈$0.3887
                                                                                Grok 4.6 · low
                                                                                Evidence and eligibility
                                                                                • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                  Figure estimate

                                                                                  60% of score weight measured. Score ≈60.3; missing-input range 52.3–64.9 points. API cost ≈$0.27 per mix task; range $0.27–$0.31.

                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                  • AA-Omniscience index · score: 58.1 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.37 points; benchmark-mean baseline 10.20.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.36.

                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  ≈60.30≈$0.269230.33$1.1439
                                                                                  Claude Fable 5 · medium
                                                                                  Evidence and eligibility

                                                                                  Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                                  Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                                  • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                  • Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
                                                                                  • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                  • Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                  • Figure · AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration
                                                                                  • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                  • Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration
                                                                                  • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                  • Execute · AutomationBench-AA partial credit: Artificial Analysis has not published a model row for this configuration
                                                                                  Figure estimate

                                                                                  0% of score weight measured. Score ≈59.8; missing-input range 44.3–71.7 points. API cost ≈$2.15 per mix task; range $0.93–$5.06.

                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                  • Briefcase analytical quality · score: 45.9 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • Briefcase analytical quality · cost: $6.76 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                                    Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AA-LCR 1.1 · score: 81.4 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AA-LCR 1.1 · cost: $0.36 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

                                                                                    Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AA-Omniscience index · score: 54.1 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AA-Omniscience index · cost: $0.023 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                                    Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  Execute estimate

                                                                                  40% of score weight measured. Score ≈47.7; missing-input range 39.3–53.9 points. API cost ≈$6.63 per mix task; range $4.40–$11.39.

                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                  • Terminal-Bench 4.0 · AA harness · score: 23.2 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • Terminal-Bench 4.0 · AA harness · cost: $10.65 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                    Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • GDPval-AA v2.1 · score: 50.7 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 43 configurations. Held-out mean absolute error 4.48 points; benchmark-mean baseline 7.08.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • GDPval-AA v2.1 · cost: $2.90 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                    Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AutomationBench-AA partial credit · score: 55.9 converted points, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18.

                                                                                    A higher or lower effort of the same model was measured, so the prediction is held at Fable 5 max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (1.9 points). The held-out test does not establish accuracy here.

                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  • AutomationBench-AA partial credit · cost: $1.16 per benchmark task, estimated.

                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                    Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                    Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                  ≈59.83≈$2.1468≈47.72≈$6.6269
                                                                                  Claude Opus 5 · low
                                                                                  Evidence and eligibility
                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                    Figure estimate

                                                                                    60% of score weight measured. Score ≈59.8; missing-input range 50.2–66.9 points. API cost ≈$0.77 per mix task; range $0.75–$0.90.

                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                    • AA-Omniscience index · score: 58.3 converted points, estimated.

                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                      Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.14 points; benchmark-mean baseline 10.01.

                                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                    • AA-Omniscience index · cost: $0.034 per benchmark task, estimated.

                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                      Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.04 log dollars; benchmark-mean baseline 1.35.

                                                                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                    ≈59.78≈$0.767144.26$2.6600
                                                                                    MiniMax M3.1 Flash · max
                                                                                    Evidence and eligibility
                                                                                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                      Execute estimate

                                                                                      60% of score weight measured. Score ≈51.9; missing-input range 48.5–55.3 points. API cost ≈$0.87 per mix task; range $0.54–$1.88.

                                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                      • DeepSWE v1.1 · mini-swe-agent · score: 67.9 converted points, estimated.

                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                      • DeepSWE v1.1 · mini-swe-agent · cost: $1.26 per scored attempt, estimated.

                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                      59.76$0.1644≈51.89≈$0.8733
                                                                                      GPT-5.5 · xhigh
                                                                                      Evidence and eligibility
                                                                                      • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                      • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                        Figure estimate

                                                                                        60% of score weight measured. Score ≈59.7; missing-input range 50.5–68.8 points. API cost ≈$1.37 per mix task; range $1.36–$1.47.

                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                        • AA-Omniscience index · score: 60.5 converted points, estimated.

                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                          Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                        • AA-Omniscience index · cost: $0.025 per benchmark task, estimated.

                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                          Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                        ≈59.66≈$1.373643.33$6.3331
                                                                                        Muse Spark 1.2 · xhigh
                                                                                        Evidence and eligibility
                                                                                        • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                        • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                          Figure estimate

                                                                                          60% of score weight measured. Score ≈59.3; missing-input range 51.4–67.3 points. API cost ≈$0.51 per mix task; range $0.51–$0.54.

                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                          • AA-Omniscience index · score: 57.2 converted points, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          • AA-Omniscience index · cost: $0.0081 per benchmark task, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          ≈59.33≈$0.514935.41$3.0244
                                                                                          Mistral Large 4 · high
                                                                                          Evidence and eligibility
                                                                                          • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                          • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                          • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                          • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                          Figure estimate

                                                                                          70% of score weight measured. Score ≈59.2; missing-input range 57.4–61.0 points. API cost ≈$0.37 per mix task; range $0.34–$0.55.

                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                          • AA-LCR 1.1 · score: 80.4 converted points, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 43 training models, 123 configurations. Held-out mean absolute error 4.00 points; benchmark-mean baseline 4.25.

                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          • AA-LCR 1.1 · cost: $0.098 per benchmark task, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: Briefcase analytical quality. 37 training models, 101 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          Execute estimate

                                                                                          50% of score weight measured. Score ≈49.3; missing-input range 44.7–53.9 points. API cost ≈$2.86 per mix task; range $2.29–$3.95.

                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 67.6 converted points, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $2.95 per scored attempt, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          • GDPval-AA v2.1 · score: 49.1 converted points, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: Briefcase analytical quality. 43 training models, 122 configurations. Held-out mean absolute error 3.51 points; benchmark-mean baseline 8.48.

                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          • GDPval-AA v2.1 · cost: $0.50 per benchmark task, estimated.

                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                            Predictor: Briefcase analytical quality. 37 training models, 99 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.27.

                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                          ≈59.23≈$0.3694≈49.31≈$2.8627
                                                                                          Qwen3.8 27B · xhigh
                                                                                          Evidence and eligibility
                                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                            Execute estimate

                                                                                            60% of score weight measured. Score ≈34.9; missing-input range 24.7–45.0 points. API cost ≈$2.94 per mix task; range $2.39–$3.95.

                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 52.6 converted points, estimated.

                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $2.94 per scored attempt, estimated.

                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                            59.17$0.9963≈34.87≈$2.9373
                                                                                            Gemini 3.8 Flash · medium59.02$1.632449.09$6.1673
                                                                                            GPT-5.5 · high
                                                                                            Evidence and eligibility
                                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                              Figure estimate

                                                                                              60% of score weight measured. Score ≈58.8; missing-input range 49.7–68.0 points. API cost ≈$1.05 per mix task; range $1.04–$1.14.

                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                              • AA-Omniscience index · score: 60.5 converted points, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • AA-Omniscience index · cost: $0.024 per benchmark task, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              ≈58.83≈$1.049739.77$4.1247
                                                                                              GPT-6 Sol · medium
                                                                                              Evidence and eligibility
                                                                                              • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                              • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                              Figure estimate

                                                                                              60% of score weight measured. Score ≈58.6; missing-input range 49.4–62.1 points. API cost ≈$0.16 per mix task; range $0.16–$0.20.

                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                              • AA-Omniscience index · score: 59.3 converted points, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.47 points; benchmark-mean baseline 10.25.

                                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.02 log dollars; benchmark-mean baseline 1.35.

                                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              Execute estimate

                                                                                              60% of score weight measured. Score ≈45.7; missing-input range 42.3–49.1 points. API cost ≈$1.01 per mix task; range $0.64–$2.12.

                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 65.5 converted points, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $1.40 per scored attempt, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              ≈58.60≈$0.1605≈45.69≈$1.0077
                                                                                              Gemini 3.8 Flash · high58.59$2.291850.14$7.8667
                                                                                              DeepSeek V4.1 Flash 1017 · high
                                                                                              Evidence and eligibility
                                                                                              • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                              • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                              • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                              • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                              Figure estimate

                                                                                              70% of score weight measured. Score ≈58.5; missing-input range 56.4–60.2 points. API cost ≈$0.13 per mix task; range $0.11–$0.21.

                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                              • AA-LCR 1.1 · score: 81.2 converted points, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 42 training models, 121 configurations. Held-out mean absolute error 3.72 points; benchmark-mean baseline 3.95.

                                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • AA-LCR 1.1 · cost: $0.054 per benchmark task, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Briefcase analytical quality. 36 training models, 99 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.12.

                                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              Execute estimate

                                                                                              50% of score weight measured. Score ≈52.7; missing-input range 48.1–57.3 points. API cost ≈$0.78 per mix task; range $0.46–$1.71.

                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 68.7 converted points, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $1.16 per scored attempt, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • GDPval-AA v2.1 · score: 48.6 converted points, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Briefcase analytical quality. 42 training models, 121 configurations. Held-out mean absolute error 3.49 points; benchmark-mean baseline 8.43.

                                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              • GDPval-AA v2.1 · cost: $0.18 per benchmark task, estimated.

                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                Predictor: Briefcase analytical quality. 36 training models, 98 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.28.

                                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                              ≈58.51≈$0.1283≈52.68≈$0.7759
                                                                                              GPT-5.6 Sol · medium
                                                                                              Evidence and eligibility
                                                                                              • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                              • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                Figure estimate

                                                                                                60% of score weight measured. Score ≈58.2; missing-input range 50.2–66.2 points. API cost ≈$0.41 per mix task; range $0.41–$0.49.

                                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                • AA-Omniscience index · score: 58.0 converted points, estimated.

                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                  Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                • AA-Omniscience index · cost: $0.019 per benchmark task, estimated.

                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                  Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                  Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                ≈58.19≈$0.413141.91$1.4701
                                                                                                MiMo-V2.6-Flash · reasoning
                                                                                                Evidence and eligibility
                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                  Execute estimate

                                                                                                  60% of score weight measured. Score ≈49.9; missing-input range 46.5–53.3 points. API cost ≈$0.33 per mix task; range $0.17–$0.82.

                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 67.0 converted points, estimated.

                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.61 per scored attempt, estimated.

                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                  57.42$0.0514≈49.94≈$0.3334
                                                                                                  GPT-6.1 Terra · low
                                                                                                  Evidence and eligibility
                                                                                                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                    Execute estimate

                                                                                                    60% of score weight measured. Score ≈42.7; missing-input range 39.3–46.1 points. API cost ≈$0.22 per mix task; range $0.076–$1.21.

                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                    • DeepSWE v1.1 · mini-swe-agent · score: 63.7 converted points, estimated.

                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                    • DeepSWE v1.1 · mini-swe-agent · cost: $0.40 per scored attempt, estimated.

                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                    56.49$0.0542≈42.73≈$0.2154
                                                                                                    GPT-6 Luna · max
                                                                                                    Evidence and eligibility
                                                                                                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                      Execute estimate

                                                                                                      60% of score weight measured. Score ≈41.9; missing-input range 32.7–51.1 points. API cost ≈$0.34 per mix task; range $0.18–$0.83.

                                                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                      • DeepSWE v1.1 · mini-swe-agent · score: 62.0 converted points, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • DeepSWE v1.1 · mini-swe-agent · cost: $0.62 per scored attempt, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      56.49$0.0550≈41.90≈$0.3434
                                                                                                      Gemini 3.7 Flash · medium
                                                                                                      Evidence and eligibility

                                                                                                      Figure excluded from recommendations: Only 30% of score weight measured; at least 50% required for a recommendation.

                                                                                                      • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                      • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                      • Figure · AA-LCR 1.1: Comparable API pricing is not published
                                                                                                      • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                      • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                      • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                      • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                      • Execute · GDPval-AA v2.1: Comparable API pricing is not published
                                                                                                      • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                      • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                      Figure estimate

                                                                                                      30% of score weight measured. Score ≈56.5; missing-input range 39.3–65.7 points. API cost ≈$0.61 per mix task; range $0.073–$5.08.

                                                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                      • Briefcase analytical quality · score: 25.7 converted points, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: AA-LCR 1.1. 42 training models, 123 configurations. Held-out mean absolute error 12.64 points; benchmark-mean baseline 14.62.

                                                                                                        A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.7 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

                                                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • Briefcase analytical quality · cost: $1.85 per benchmark task, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                                                        Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • AA-LCR 1.1 · cost: $0.16 per benchmark task, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 0.95.

                                                                                                        Missing measurement: Comparable API pricing is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • AA-Omniscience index · score: 59.7 converted points, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • AA-Omniscience index · cost: $0.0084 per benchmark task, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                                                        Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      Execute estimate

                                                                                                      50% of score weight measured. Score ≈44.4; missing-input range 37.2–46.3 points. API cost ≈$1.84 per mix task; range $0.96–$7.93.

                                                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                      • Terminal-Bench 4.0 · AA harness · score: 15.8 converted points, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 42 training models, 120 configurations. Held-out mean absolute error 11.02 points; benchmark-mean baseline 16.94.

                                                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • Terminal-Bench 4.0 · AA harness · cost: $2.55 per benchmark task, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                                        Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • GDPval-AA v2.1 · cost: $0.80 per benchmark task, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                                        Missing measurement: Comparable API pricing is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • AutomationBench-AA partial credit · score: 55.8 converted points, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 5.92 points; benchmark-mean baseline 10.39.

                                                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      • AutomationBench-AA partial credit · cost: $0.38 per benchmark task, estimated.

                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                        Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                                        Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                      ≈56.48≈$0.6059≈44.42≈$1.8415
                                                                                                      GPT-5.5 · medium
                                                                                                      Evidence and eligibility
                                                                                                      • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                      • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                        Figure estimate

                                                                                                        60% of score weight measured. Score ≈56.3; missing-input range 47.1–65.5 points. API cost ≈$0.63 per mix task; range $0.63–$0.72.

                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                        • AA-Omniscience index · score: 59.7 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AA-Omniscience index · cost: $0.024 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        ≈56.28≈$0.634233.09$2.1564
                                                                                                        Claude Fable 5 · low
                                                                                                        Evidence and eligibility

                                                                                                        Figure excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                                                        Execute excluded from recommendations: Fallback behavior is not disclosed; exact system cannot be established.

                                                                                                        • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                        • Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
                                                                                                        • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                                        • Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
                                                                                                        • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                        • Figure · AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
                                                                                                        • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                        • Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration
                                                                                                        • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                        • Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration
                                                                                                        • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                        • Execute · AutomationBench-AA partial credit: Artificial Analysis has not published a model row for this configuration
                                                                                                        Figure estimate

                                                                                                        0% of score weight measured. Score ≈56.0; missing-input range 41.7–68.7 points. API cost ≈$1.23 per mix task; range $0.46–$3.40.

                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                        • Briefcase analytical quality · score: 41.3 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • Briefcase analytical quality · cost: $3.83 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                                                          Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AA-LCR 1.1 · score: 80.4 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AA-LCR 1.1 · cost: $0.25 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

                                                                                                          Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AA-Omniscience index · score: 48.7 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AA-Omniscience index · cost: $0.015 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                                                          Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        Execute estimate

                                                                                                        40% of score weight measured. Score ≈42.3; missing-input range 34.8–48.3 points. API cost ≈$3.77 per mix task; range $2.29–$8.05.

                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                        • Terminal-Bench 4.0 · AA harness · score: 16.4 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • Terminal-Bench 4.0 · AA harness · cost: $5.70 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                                          Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • GDPval-AA v2.1 · score: 48.3 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 43 configurations. Held-out mean absolute error 4.48 points; benchmark-mean baseline 7.08.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • GDPval-AA v2.1 · cost: $1.65 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                                          Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AutomationBench-AA partial credit · score: 52.4 converted points, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18.

                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        • AutomationBench-AA partial credit · cost: $0.71 per benchmark task, estimated.

                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                          Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                                          Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                        ≈56.02≈$1.2298≈42.29≈$3.7685
                                                                                                        DeepSeek V4 Pro 0813 · max
                                                                                                        Evidence and eligibility
                                                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                          Execute estimate

                                                                                                          60% of score weight measured. Score ≈43.5; missing-input range 40.1–46.9 points. API cost ≈$2.48 per mix task; range $1.99–$3.40.

                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 63.0 converted points, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $2.65 per scored attempt, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          55.90$0.3450≈43.46≈$2.4829
                                                                                                          GPT-5.6 Terra · max55.65$0.943353.98$4.4476
                                                                                                          Gemini 3.7 Flash · high
                                                                                                          Evidence and eligibility
                                                                                                          • Figure · Briefcase analytical quality: Cache-hit accounting is not published
                                                                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                          • Execute · Terminal-Bench 4.0 · AA harness: Cache-hit accounting is not published
                                                                                                          • Execute · GDPval-AA v2.1: Cache-hit accounting is not published
                                                                                                          • Execute · AutomationBench-AA partial credit: Cache-hit accounting is not published
                                                                                                          Figure estimate

                                                                                                          60% of score weight measured. Score ≈55.6; missing-input range 46.5–64.8 points. API cost ≈$0.42 per mix task; range $0.064–$4.34.

                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                          • Briefcase analytical quality · cost: $1.32 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 37 training models, 101 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                            Missing measurement: Cache-hit accounting is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AA-Omniscience index · score: 58.9 converted points, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AA-Omniscience index · cost: $0.0058 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          Execute estimate

                                                                                                          100% of score weight measured. Score 44.7; no score component estimated. API cost ≈$2.00 per mix task; range $1.04–$8.68.

                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                          • Terminal-Bench 4.0 · AA harness · cost: $2.80 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                                            Missing measurement: Cache-hit accounting is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • GDPval-AA v2.1 · cost: $0.87 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                                            Missing measurement: Cache-hit accounting is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AutomationBench-AA partial credit · cost: $0.41 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                                            Missing measurement: Cache-hit accounting is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          ≈55.63≈$0.424844.70≈$2.0008
                                                                                                          GPT-5.5 · low
                                                                                                          Evidence and eligibility

                                                                                                          Figure excluded from recommendations: Only 30% of score weight measured; at least 50% required for a recommendation.

                                                                                                          • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                          • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                          • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                          • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                          • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                          • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                          Figure estimate

                                                                                                          30% of score weight measured. Score ≈55.3; missing-input range 36.9–65.0 points. API cost ≈$1.82 per mix task; range $0.38–$12.02.

                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                          • Briefcase analytical quality · score: 25.4 converted points, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 42 training models, 121 configurations. Held-out mean absolute error 12.68 points; benchmark-mean baseline 14.76.

                                                                                                            A higher or lower effort of the same model was measured, so the prediction is held at GPT-5.5 medium’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • Briefcase analytical quality · cost: $5.56 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 36 training models, 98 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.37.

                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AA-Omniscience index · score: 58.5 converted points, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AA-Omniscience index · cost: $0.023 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          Execute estimate

                                                                                                          50% of score weight measured. Score ≈19.6; missing-input range 16.4–23.0 points. API cost ≈$0.80 per mix task; range $0.57–$1.85.

                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                          • Terminal-Bench 4.0 · AA harness · score: 3.8 converted points, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 42 training models, 118 configurations. Held-out mean absolute error 11.04 points; benchmark-mean baseline 16.85.

                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • Terminal-Bench 4.0 · AA harness · cost: $0.77 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: GDPval-AA v2.1. 35 training models, 95 configurations. Held-out mean absolute error 0.44 log dollars; benchmark-mean baseline 1.29.

                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AutomationBench-AA partial credit · score: 31.7 converted points, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 37 configurations. Held-out mean absolute error 5.50 points; benchmark-mean baseline 10.05.

                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          • AutomationBench-AA partial credit · cost: $0.22 per benchmark task, estimated.

                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                            Predictor: DeepSWE v1.1 · mini-swe-agent. 12 training models, 36 configurations. Held-out mean absolute error 0.71 log dollars; benchmark-mean baseline 0.91.

                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                          ≈55.30≈$1.8214≈19.64≈$0.8040
                                                                                                          GPT-5.6 Luna · xhigh
                                                                                                          Evidence and eligibility
                                                                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                            Figure estimate

                                                                                                            60% of score weight measured. Score ≈55.1; missing-input range 46.2–55.1 points. API cost ≈$0.071 per mix task; range $0.070–$0.085.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • AA-Omniscience index · score: 48.7 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 8.87 points; benchmark-mean baseline 9.78.

                                                                                                              A higher or lower effort of the same model was measured, so the prediction is held at Luna max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (7.2 points). The held-out test does not establish accuracy here.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-Omniscience index · cost: $0.0014 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.38.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            ≈55.11≈$0.070634.91$0.7374
                                                                                                            Mistral Large 4 · medium
                                                                                                            Evidence and eligibility
                                                                                                            • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                                            • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                            • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                            • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                                            Figure estimate

                                                                                                            70% of score weight measured. Score ≈55.1; missing-input range 48.4–61.1 points. API cost ≈$0.15 per mix task; range $0.14–$0.23.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • AA-LCR 1.1 · score: 79.9 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 43 training models, 123 configurations. Held-out mean absolute error 4.00 points; benchmark-mean baseline 4.25.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-LCR 1.1 · cost: $0.057 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Briefcase analytical quality. 37 training models, 101 configurations. Held-out mean absolute error 0.86 log dollars; benchmark-mean baseline 1.13.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            Execute estimate

                                                                                                            50% of score weight measured. Score ≈47.2; missing-input range 42.6–51.8 points. API cost ≈$1.97 per mix task; range $1.54–$2.78.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 67.2 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $2.31 per scored attempt, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • GDPval-AA v2.1 · score: 43.0 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Briefcase analytical quality. 43 training models, 122 configurations. Held-out mean absolute error 3.51 points; benchmark-mean baseline 8.48.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • GDPval-AA v2.1 · cost: $0.21 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Briefcase analytical quality. 37 training models, 99 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.27.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            ≈55.09≈$0.1508≈47.23≈$1.9718
                                                                                                            Qwen3.8 Max · xhigh
                                                                                                            Evidence and eligibility

                                                                                                            Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation.

                                                                                                            Execute excluded from recommendations: Only 40% of score weight measured; at least 50% required for a recommendation.

                                                                                                            • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                            • Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
                                                                                                            • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                                            • Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
                                                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                            • Figure · AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration
                                                                                                            • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                            • Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration
                                                                                                            • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                            • Execute · AutomationBench-AA partial credit: Artificial Analysis has not published a model row for this configuration
                                                                                                            Figure estimate

                                                                                                            0% of score weight measured. Score ≈54.7; missing-input range 40.4–69.1 points. API cost ≈$1.22 per mix task; range $0.45–$3.37.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • Briefcase analytical quality · score: 39.7 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • Briefcase analytical quality · cost: $3.80 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-LCR 1.1 · score: 80.1 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-LCR 1.1 · cost: $0.24 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-Omniscience index · score: 47.0 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-Omniscience index · cost: $0.015 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            Execute estimate

                                                                                                            40% of score weight measured. Score ≈40.4; missing-input range 32.9–47.8 points. API cost ≈$3.73 per mix task; range $2.27–$7.97.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • Terminal-Bench 4.0 · AA harness · score: 14.3 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • Terminal-Bench 4.0 · AA harness · cost: $5.64 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • GDPval-AA v2.1 · score: 47.3 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 43 configurations. Held-out mean absolute error 4.48 points; benchmark-mean baseline 7.08.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • GDPval-AA v2.1 · cost: $1.63 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AutomationBench-AA partial credit · score: 51.1 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AutomationBench-AA partial credit · cost: $0.71 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            ≈54.72≈$1.2189≈40.37≈$3.7349
                                                                                                            GPT-5.6 Luna · max54.35$0.128943.31$1.5332
                                                                                                            Gemini 3.7 Flash · low
                                                                                                            Evidence and eligibility

                                                                                                            Figure excluded from recommendations: Only 30% of score weight measured; at least 50% required for a recommendation.

                                                                                                            • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                            • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                            • Figure · AA-LCR 1.1: Comparable API pricing is not published
                                                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                            • Execute · GDPval-AA v2.1: Comparable API pricing is not published
                                                                                                            • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                            • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                            Figure estimate

                                                                                                            30% of score weight measured. Score ≈54.1; missing-input range 37.2–62.1 points. API cost ≈$0.54 per mix task; range $0.063–$4.70.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • Briefcase analytical quality · score: 25.7 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: AA-LCR 1.1. 42 training models, 123 configurations. Held-out mean absolute error 12.64 points; benchmark-mean baseline 14.62.

                                                                                                              A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.7 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • Briefcase analytical quality · cost: $1.65 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-LCR 1.1 · cost: $0.14 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.82 log dollars; benchmark-mean baseline 0.95.

                                                                                                              Missing measurement: Comparable API pricing is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-Omniscience index · score: 57.0 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AA-Omniscience index · cost: $0.0077 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            Execute estimate

                                                                                                            50% of score weight measured. Score ≈37.7; missing-input range 30.8–41.1 points. API cost ≈$1.64 per mix task; range $0.84–$8.44.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • Terminal-Bench 4.0 · AA harness · score: 13.7 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 42 training models, 120 configurations. Held-out mean absolute error 11.02 points; benchmark-mean baseline 16.94.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • Terminal-Bench 4.0 · AA harness · cost: $2.24 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • GDPval-AA v2.1 · cost: $0.71 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                                              Missing measurement: Comparable API pricing is not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AutomationBench-AA partial credit · score: 48.3 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 5.92 points; benchmark-mean baseline 10.39.

                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • AutomationBench-AA partial credit · cost: $0.35 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            ≈54.10≈$0.5403≈37.66≈$1.6411
                                                                                                            GPT-5.6 Terra · xhigh53.88$0.608239.45$1.5608
                                                                                                            Gemini 3.8 Flash · low
                                                                                                            Evidence and eligibility
                                                                                                            • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                            Figure estimate

                                                                                                            100% of score weight measured. Score 53.4; no score component estimated. API cost ≈$0.38 per mix task; range $0.057–$4.07.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • Briefcase analytical quality · cost: $1.19 per benchmark task, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: AA-LCR 1.1. 36 training models, 99 configurations. Held-out mean absolute error 1.00 log dollars; benchmark-mean baseline 1.35.

                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            Execute estimate

                                                                                                            60% of score weight measured. Score ≈36.8; missing-input range 28.0–42.4 points. API cost ≈$9.24 per mix task; range $7.75–$12.34.

                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 59.2 converted points, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 13 training models, 38 configurations. Held-out mean absolute error 6.97 points; benchmark-mean baseline 11.99.

                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $7.24 per scored attempt, estimated.

                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness. 12 training models, 37 configurations. Held-out mean absolute error 0.46 log dollars; benchmark-mean baseline 0.82.

                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                            53.41≈$0.3835≈36.76≈$9.2432
                                                                                                            GPT-5.6 Sol · low
                                                                                                            Evidence and eligibility
                                                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                              Figure estimate

                                                                                                              60% of score weight measured. Score ≈53.1; missing-input range 43.4–62.8 points. API cost ≈$0.25 per mix task; range $0.24–$0.32.

                                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                              • AA-Omniscience index · score: 56.6 converted points, estimated.

                                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                              • AA-Omniscience index · cost: $0.019 per benchmark task, estimated.

                                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                              ≈53.10≈$0.247828.71$0.7105
                                                                                                              GPT-5.6 Luna · high
                                                                                                              Evidence and eligibility
                                                                                                              • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                              • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                Figure estimate

                                                                                                                60% of score weight measured. Score ≈53.0; missing-input range 45.2–53.0 points. API cost ≈$0.041 per mix task; range $0.041–$0.055.

                                                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                • AA-Omniscience index · score: 48.7 converted points, estimated.

                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                  Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 8.87 points; benchmark-mean baseline 9.78.

                                                                                                                  A higher or lower effort of the same model was measured, so the prediction is held at Luna max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (7.2 points). The held-out test does not establish accuracy here.

                                                                                                                  Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • AA-Omniscience index · cost: $0.0014 per benchmark task, estimated.

                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                  Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.38.

                                                                                                                  Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                ≈52.98≈$0.041428.11$0.3613
                                                                                                                Claude Haiku 5.5 · low
                                                                                                                Evidence and eligibility
                                                                                                                • Figure · Briefcase analytical quality: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                                                                • Figure · AA-LCR 1.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                                                                • Figure · AA-Omniscience index: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                • Execute · Terminal-Bench 4.0 · AA harness: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                                                                • Execute · GDPval-AA v2.1: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                                                                • Execute · AutomationBench-AA partial credit: AA cost accounting omits higher long-context pricing; request-tier allocations are not published
                                                                                                                Figure estimate

                                                                                                                100% of score weight measured. Score 52.9; no score component estimated. API cost ≈$0.056 per mix task; conditional tier envelope $0.019–$0.093.

                                                                                                                Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it.

                                                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                • Briefcase analytical quality · cost: $0.14 per benchmark task, estimated.

                                                                                                                  Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                                                  Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • AA-LCR 1.1 · cost: $0.044 per benchmark task, estimated.

                                                                                                                  Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                                                  Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • AA-Omniscience index · cost: $0.00076 per benchmark task, estimated.

                                                                                                                  Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                                                  Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                Execute estimate

                                                                                                                60% of score weight measured. Score ≈35.8; missing-input range 26.5–45.0 points. API cost ≈$0.44 per mix task; range unknown, because part of the cost is an accounting scenario.

                                                                                                                Request-tier scenario: the midpoint sets the displayed position; it is not an expected price. The tier envelope assumes the captured tokens, cache-hit rate, five-minute cache writes and task counts. Other billing modifiers are outside it. The missing benchmark cost is predicted separately; the total bill has no established bound.

                                                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                • DeepSWE v1.1 · mini-swe-agent · score: 62.0 converted points, estimated.

                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • DeepSWE v1.1 · mini-swe-agent · cost: $0.76 per scored attempt, estimated.

                                                                                                                  Predicted from the named predictor benchmark’s conditional request-tier midpoint scenario, not a measured bill. Regression trains on measured component pairs with whole models held out. The source accounting and transfer errors are not jointly bounded; the actual bill and range are unknown.

                                                                                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • Terminal-Bench 4.0 · AA harness · cost: $0.36 per benchmark task, estimated.

                                                                                                                  Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                                                  Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • GDPval-AA v2.1 · cost: $0.031 per benchmark task, estimated.

                                                                                                                  Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                                                  Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                • AutomationBench-AA partial credit · cost: $0.046 per benchmark task, estimated.

                                                                                                                  Midpoint of all-base and all-premium request-tier scenarios above 100,000 prompt tokens, including cached tokens. Conditional on captured AA tokens, cache-hit rate, five-minute cache writes and scored-task denominators; not an expected bill, guaranteed actual-bill bound or confidence interval. Request-tier allocations and other billing modifiers are unknown. Tariffs: https://platform.claude.com/docs/en/about-claude/pricing (checked 2026-10-08).

                                                                                                                  Missing measurement: AA cost accounting omits higher long-context pricing; request-tier allocations are not published. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                52.91≈$0.0561midpoint scenario · conditional $0.0187–$0.0935≈35.79≈$0.4402scenario + prediction · bill range unknown
                                                                                                                DeepSeek V4.1 Flash 1017 · none
                                                                                                                Evidence and eligibility
                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                  • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                                  • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                                                  Execute estimate

                                                                                                                  50% of score weight measured. Score ≈39.0; missing-input range 29.1–48.9 points. API cost ≈$0.57 per mix task; range $0.31–$1.33.

                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 58.5 converted points, estimated.

                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.90 per scored attempt, estimated.

                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                  • GDPval-AA v2.1 · score: 50.8 converted points, estimated.

                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                    Predictor: Briefcase analytical quality. 42 training models, 121 configurations. Held-out mean absolute error 3.49 points; benchmark-mean baseline 8.43.

                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                  • GDPval-AA v2.1 · cost: $0.37 per benchmark task, estimated.

                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                    Predictor: Briefcase analytical quality. 36 training models, 98 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.28.

                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                  52.80$0.2535≈39.00≈$0.5709
                                                                                                                  GPT-6 Luna · xhigh
                                                                                                                  Evidence and eligibility
                                                                                                                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                    Execute estimate

                                                                                                                    60% of score weight measured. Score ≈37.2; missing-input range 28.0–46.5 points. API cost ≈$0.30 per mix task; range $0.15–$0.75.

                                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                    • DeepSWE v1.1 · mini-swe-agent · score: 57.2 converted points, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    • DeepSWE v1.1 · mini-swe-agent · cost: $0.56 per scored attempt, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    52.72$0.0280≈37.24≈$0.3020
                                                                                                                    GPT-6 Sol · low
                                                                                                                    Evidence and eligibility
                                                                                                                    • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                    • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                    Figure estimate

                                                                                                                    60% of score weight measured. Score ≈52.6; missing-input range 44.6–56.8 points. API cost ≈$0.099 per mix task; range $0.096–$0.14.

                                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                    • AA-Omniscience index · score: 57.4 converted points, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.47 points; benchmark-mean baseline 10.25.

                                                                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.02 log dollars; benchmark-mean baseline 1.35.

                                                                                                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    Execute estimate

                                                                                                                    60% of score weight measured. Score ≈38.2; missing-input range 28.9–47.4 points. API cost ≈$0.57 per mix task; range $0.33–$1.29.

                                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                    • DeepSWE v1.1 · mini-swe-agent · score: 58.5 converted points, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    • DeepSWE v1.1 · mini-swe-agent · cost: $0.90 per scored attempt, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    ≈52.60≈$0.0993≈38.18≈$0.5701
                                                                                                                    Ling 3.0 Flash VL · default (assumed)
                                                                                                                    Evidence and eligibility
                                                                                                                    • Figure · Briefcase analytical quality: Comparable API pricing is not published
                                                                                                                    • Figure · AA-LCR 1.1: Comparable API pricing is not published
                                                                                                                    • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                    • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                    • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                    • Execute · Terminal-Bench 4.0 · AA harness: Comparable API pricing is not published
                                                                                                                    • Execute · GDPval-AA v2.1: Comparable API pricing is not published
                                                                                                                    • Execute · AutomationBench-AA partial credit: Comparable API pricing is not published
                                                                                                                    Figure estimate

                                                                                                                    60% of score weight measured. Score ≈52.3; missing-input range 42.5–62.0 points. API cost unknown.

                                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                    • AA-Omniscience index · score: 56.8 converted points, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    Execute estimate

                                                                                                                    60% of score weight measured. Score ≈7.4; missing-input range 5.7–17.5 points. API cost unknown.

                                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                    • DeepSWE v1.1 · mini-swe-agent · score: 4.2 converted points, estimated.

                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                      Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                      Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                    ≈52.26Unpriced≈7.36Unpriced
                                                                                                                    Qwen3.8 27B · medium
                                                                                                                    Evidence and eligibility
                                                                                                                      • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                      Execute estimate

                                                                                                                      60% of score weight measured. Score ≈32.8; missing-input range 22.7–43.0 points. API cost ≈$3.85 per mix task; range $3.18–$5.11.

                                                                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                      • DeepSWE v1.1 · mini-swe-agent · score: 51.4 converted points, estimated.

                                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                        Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                      • DeepSWE v1.1 · mini-swe-agent · cost: $3.55 per scored attempt, estimated.

                                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                        Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                        Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                      52.19$0.9658≈32.82≈$3.8500
                                                                                                                      Qwen3.8 27B · low
                                                                                                                      Evidence and eligibility
                                                                                                                        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                        Execute estimate

                                                                                                                        60% of score weight measured. Score ≈26.5; missing-input range 16.4–36.7 points. API cost ≈$4.03 per mix task; range $3.34–$5.32.

                                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                        • DeepSWE v1.1 · mini-swe-agent · score: 42.0 converted points, estimated.

                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                        • DeepSWE v1.1 · mini-swe-agent · cost: $3.65 per scored attempt, estimated.

                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                          Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                        51.81$0.6995≈26.54≈$4.0274
                                                                                                                        GLM-5.3 · low
                                                                                                                        Evidence and eligibility
                                                                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                          Execute estimate

                                                                                                                          60% of score weight measured. Score ≈52.2; missing-input range 48.1–52.7 points. API cost ≈$3.71 per mix task; range $2.95–$5.40.

                                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 70.1 converted points, estimated.

                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 7.11 points; benchmark-mean baseline 12.19.

                                                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $3.51 per scored attempt, estimated.

                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 12 training models, 38 configurations. Held-out mean absolute error 0.58 log dollars; benchmark-mean baseline 0.81.

                                                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                          51.73$0.2627≈52.21≈$3.7137
                                                                                                                          GPT-6 Luna · high
                                                                                                                          Evidence and eligibility
                                                                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                            Execute estimate

                                                                                                                            60% of score weight measured. Score ≈33.0; missing-input range 22.9–43.2 points. API cost ≈$0.22 per mix task; range $0.073–$1.29.

                                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 50.0 converted points, estimated.

                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $0.43 per scored attempt, estimated.

                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                            51.38$0.0218≈33.04≈$0.2211
                                                                                                                            GPT-5.6 Terra · high51.00$0.319132.77$0.7875
                                                                                                                            Kimi K3 · low
                                                                                                                            Evidence and eligibility
                                                                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                              Execute estimate

                                                                                                                              60% of score weight measured. Score ≈39.1; missing-input range 30.6–43.3 points. API cost ≈$5.06 per mix task; range $3.93–$8.58.

                                                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 60.4 converted points, estimated.

                                                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 6.98 points; benchmark-mean baseline 12.22.

                                                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $4.14 per scored attempt, estimated.

                                                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 12 training models, 38 configurations. Held-out mean absolute error 0.55 log dollars; benchmark-mean baseline 0.80.

                                                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                              50.88$0.2521≈39.14≈$5.0615
                                                                                                                              GPT-6.1 Terra · none
                                                                                                                              Evidence and eligibility
                                                                                                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                Execute estimate

                                                                                                                                60% of score weight measured. Score ≈48.2; missing-input range 44.8–51.6 points. API cost ≈$0.47 per mix task; range $0.26–$1.10.

                                                                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                • DeepSWE v1.1 · mini-swe-agent · score: 68.2 converted points, estimated.

                                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                • DeepSWE v1.1 · mini-swe-agent · cost: $0.79 per scored attempt, estimated.

                                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                50.86$0.0945≈48.23≈$0.4709
                                                                                                                                GPT-6 Luna · medium
                                                                                                                                Evidence and eligibility
                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                  Execute estimate

                                                                                                                                  60% of score weight measured. Score ≈27.6; missing-input range 17.4–37.8 points. API cost ≈$0.16 per mix task; range $0.048–$0.97.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 42.0 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.33 per scored attempt, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  49.18$0.0130≈27.59≈$0.1605
                                                                                                                                  GPT-5.5 · none
                                                                                                                                  Evidence and eligibility

                                                                                                                                  Figure excluded from recommendations: Only 30% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                  Execute excluded from recommendations: Only 10% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                  • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                                                  • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                                                  • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                                                  • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                                                  Figure estimate

                                                                                                                                  30% of score weight measured. Score ≈47.9; missing-input range 29.5–57.8 points. API cost ≈$1.80 per mix task; range $0.38–$11.87.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • Briefcase analytical quality · score: 31.8 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 42 training models, 121 configurations. Held-out mean absolute error 12.68 points; benchmark-mean baseline 14.76.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • Briefcase analytical quality · cost: $5.49 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 36 training models, 98 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.37.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AA-Omniscience index · score: 47.8 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AA-Omniscience index · cost: $0.023 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  Execute estimate

                                                                                                                                  10% of score weight measured. Score ≈5.3; missing-input range 2.4–24.5 points. API cost ≈$0.54 per mix task; range $0.15–$2.07.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 1.5 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 12.40.

                                                                                                                                    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.78 per scored attempt, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: GDPval-AA v2.1. 12 training models, 36 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.80.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • Terminal-Bench 4.0 · AA harness · score: 2.3 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 42 training models, 118 configurations. Held-out mean absolute error 11.04 points; benchmark-mean baseline 16.85.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • Terminal-Bench 4.0 · AA harness · cost: $0.58 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: GDPval-AA v2.1. 35 training models, 95 configurations. Held-out mean absolute error 0.44 log dollars; benchmark-mean baseline 1.29.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AutomationBench-AA partial credit · score: 10.2 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 42 training models, 119 configurations. Held-out mean absolute error 8.23 points; benchmark-mean baseline 11.40.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AutomationBench-AA partial credit · cost: $0.077 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: GDPval-AA v2.1. 36 training models, 97 configurations. Held-out mean absolute error 0.61 log dollars; benchmark-mean baseline 1.15.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  ≈47.88≈$1.7991≈5.34≈$0.5406
                                                                                                                                  GPT-6 Sol · none
                                                                                                                                  Evidence and eligibility
                                                                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                  Figure estimate

                                                                                                                                  60% of score weight measured. Score ≈47.8; missing-input range 39.8–55.8 points. API cost ≈$0.17 per mix task; range $0.17–$0.21.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • AA-Omniscience index · score: 47.9 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 9.47 points; benchmark-mean baseline 10.25.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.02 log dollars; benchmark-mean baseline 1.35.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  Execute estimate

                                                                                                                                  60% of score weight measured. Score ≈38.4; missing-input range 29.2–47.7 points. API cost ≈$1.49 per mix task; range $0.99–$2.98.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 62.3 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $1.87 per scored attempt, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  ≈47.81≈$0.1731≈38.43≈$1.4914
                                                                                                                                  Gemini 3.5 Flash · medium
                                                                                                                                  Evidence and eligibility

                                                                                                                                  Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                                                  • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                                                  • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                                                                  • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                                                  • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                                                  Figure estimate

                                                                                                                                  60% of score weight measured. Score ≈47.3; missing-input range 39.4–55.3 points. API cost ≈$2.59 per mix task; range $2.59–$2.63.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • AA-Omniscience index · score: 54.3 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  Execute estimate

                                                                                                                                  0% of score weight measured. Score ≈10.8; missing-input range 2.0–28.7 points. API cost ≈$6.46 per mix task; range $3.10–$13.80.

                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 13.6 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality, on a curve that levels off near the top. 13 training models, 39 configurations. Held-out mean absolute error 8.19 points; benchmark-mean baseline 9.83.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $5.47 per scored attempt, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality. 12 training models, 38 configurations. Held-out mean absolute error 0.52 log dollars; benchmark-mean baseline 0.82.

                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • Terminal-Bench 4.0 · AA harness · score: 0.0 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality. 45 training models, 130 configurations. Held-out mean absolute error 10.74 points; benchmark-mean baseline 16.01.

                                                                                                                                    The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • Terminal-Bench 4.0 · AA harness · cost: $10.91 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality. 39 training models, 111 configurations. Held-out mean absolute error 0.46 log dollars; benchmark-mean baseline 1.26.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • GDPval-AA v2.1 · score: 28.5 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality, on a curve that levels off near the top. 42 training models, 121 configurations. Held-out mean absolute error 3.39 points; benchmark-mean baseline 8.26.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • GDPval-AA v2.1 · cost: $3.20 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality. 36 training models, 98 configurations. Held-out mean absolute error 0.22 log dollars; benchmark-mean baseline 1.27.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AutomationBench-AA partial credit · score: 17.0 converted points, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality, on a curve that levels off near the top. 45 training models, 131 configurations. Held-out mean absolute error 8.41 points; benchmark-mean baseline 10.95.

                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  • AutomationBench-AA partial credit · cost: $0.86 per benchmark task, estimated.

                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                    Predictor: Briefcase analytical quality. 40 training models, 113 configurations. Held-out mean absolute error 0.66 log dollars; benchmark-mean baseline 1.19.

                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                  ≈47.32≈$2.5890≈10.84≈$6.4555
                                                                                                                                  GPT-5.6 Luna · medium
                                                                                                                                  Evidence and eligibility
                                                                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                    Figure estimate

                                                                                                                                    60% of score weight measured. Score ≈46.7; missing-input range 38.9–46.7 points. API cost ≈$0.015 per mix task; range $0.015–$0.029.

                                                                                                                                    Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                    • AA-Omniscience index · score: 48.7 converted points, estimated.

                                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                      Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 8.87 points; benchmark-mean baseline 9.78.

                                                                                                                                      A higher or lower effort of the same model was measured, so the prediction is held at Luna max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (7.2 points). The held-out test does not establish accuracy here.

                                                                                                                                      Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                    • AA-Omniscience index · cost: $0.0014 per benchmark task, estimated.

                                                                                                                                      Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                      Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.38.

                                                                                                                                      Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                    ≈46.66≈$0.015111.34$0.1012
                                                                                                                                    Gemini 3.5 Flash · high
                                                                                                                                    Evidence and eligibility
                                                                                                                                    • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                    • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                      Figure estimate

                                                                                                                                      60% of score weight measured. Score ≈46.7; missing-input range 38.7–54.6 points. API cost ≈$2.36 per mix task; range $2.36–$2.40.

                                                                                                                                      Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                      • AA-Omniscience index · score: 53.7 converted points, estimated.

                                                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                        Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                                        Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                      • AA-Omniscience index · cost: $0.011 per benchmark task, estimated.

                                                                                                                                        Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                        Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                        Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                      ≈46.65≈$2.360726.59$6.6802
                                                                                                                                      GPT-5.6 Terra · medium46.52$0.174423.30$0.4181
                                                                                                                                      Qwen3.8 27B · none
                                                                                                                                      Evidence and eligibility
                                                                                                                                        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                        Execute estimate

                                                                                                                                        60% of score weight measured. Score ≈6.4; missing-input range 4.8–16.6 points. API cost ≈$5.21 per mix task; range $4.03–$9.07.

                                                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                        • DeepSWE v1.1 · mini-swe-agent · score: 4.2 converted points, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        • DeepSWE v1.1 · mini-swe-agent · cost: $4.29 per scored attempt, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        44.74$2.7150≈6.44≈$5.2140
                                                                                                                                        GPT-5.6 Terra · low44.68$0.148717.60$0.3068
                                                                                                                                        GPT-5.6 Sol · none
                                                                                                                                        Evidence and eligibility

                                                                                                                                        Execute excluded from recommendations: Only 25% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                        • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                        • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                        • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                        • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                                                        • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                                                        Figure estimate

                                                                                                                                        60% of score weight measured. Score ≈44.2; missing-input range 36.3–52.2 points. API cost ≈$0.26 per mix task; range $0.25–$0.33.

                                                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                        • AA-Omniscience index · score: 46.8 converted points, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        • AA-Omniscience index · cost: $0.019 per benchmark task, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        Execute estimate

                                                                                                                                        25% of score weight measured. Score ≈16.8; missing-input range 7.1–25.4 points. API cost ≈$0.84 per mix task; range $0.30–$2.85.

                                                                                                                                        Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                        • DeepSWE v1.1 · mini-swe-agent · score: 15.2 converted points, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: AutomationBench-AA partial credit, on a curve that levels off near the top. 13 training models, 35 configurations. Held-out mean absolute error 6.56 points; benchmark-mean baseline 12.06.

                                                                                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        • DeepSWE v1.1 · mini-swe-agent · cost: $1.05 per scored attempt, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: GDPval-AA v2.1. 12 training models, 35 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.80.

                                                                                                                                          Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        • Terminal-Bench 4.0 · AA harness · score: 10.1 converted points, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: GDPval-AA v2.1, on a curve that levels off near the top. 42 training models, 116 configurations. Held-out mean absolute error 10.97 points; benchmark-mean baseline 16.98.

                                                                                                                                          Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        • Terminal-Bench 4.0 · AA harness · cost: $0.93 per benchmark task, estimated.

                                                                                                                                          Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                          Predictor: GDPval-AA v2.1. 35 training models, 93 configurations. Held-out mean absolute error 0.43 log dollars; benchmark-mean baseline 1.29.

                                                                                                                                          Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                        ≈44.22≈$0.2594≈16.78≈$0.8351
                                                                                                                                        GPT-5.6 Luna · low
                                                                                                                                        Evidence and eligibility
                                                                                                                                        • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                        • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                          Figure estimate

                                                                                                                                          60% of score weight measured. Score ≈41.9; missing-input range 34.1–41.9 points. API cost ≈$0.010 per mix task; range $0.0095–$0.024.

                                                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                          • AA-Omniscience index · score: 48.7 converted points, estimated.

                                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                            Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 8.87 points; benchmark-mean baseline 9.78.

                                                                                                                                            A higher or lower effort of the same model was measured, so the prediction is held at Luna max’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (7.2 points). The held-out test does not establish accuracy here.

                                                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                          • AA-Omniscience index · cost: $0.0014 per benchmark task, estimated.

                                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                            Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.38.

                                                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                          ≈41.87≈$0.01014.88$0.0365
                                                                                                                                          DeepSeek V4.1 Flash · none
                                                                                                                                          Evidence and eligibility
                                                                                                                                          • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                          • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                          • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                          Figure estimate

                                                                                                                                          60% of score weight measured. Score ≈41.3; missing-input range 33.3–49.2 points. API cost ≈$0.065 per mix task; range $0.064–$0.091.

                                                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                          • AA-Omniscience index · score: 42.4 converted points, estimated.

                                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                            Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                                            Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                          • AA-Omniscience index · cost: $0.0023 per benchmark task, estimated.

                                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                            Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                            Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                          Execute estimate

                                                                                                                                          60% of score weight measured. Score ≈34.4; missing-input range 24.3–44.6 points. API cost ≈$1.24 per mix task; range $0.79–$2.56.

                                                                                                                                          Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                          • DeepSWE v1.1 · mini-swe-agent · score: 52.6 converted points, estimated.

                                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                            Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                          • DeepSWE v1.1 · mini-swe-agent · cost: $1.67 per scored attempt, estimated.

                                                                                                                                            Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                            Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                            Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                          ≈41.27≈$0.0646≈34.42≈$1.2379
                                                                                                                                          GPT-6 Luna · low
                                                                                                                                          Evidence and eligibility
                                                                                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                            Execute estimate

                                                                                                                                            60% of score weight measured. Score ≈6.2; missing-input range 4.5–16.3 points. API cost ≈$0.054 per mix task; range $0.0079–$0.55.

                                                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 4.2 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $0.13 per scored attempt, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                              The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            40.87$0.0048≈6.17≈$0.0539
                                                                                                                                            Gemini 3.5 Flash · minimal
                                                                                                                                            Evidence and eligibility

                                                                                                                                            Figure excluded from recommendations: Only 30% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                            Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                            • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                                                            • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                            • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                            • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                                                            • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                                                            • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                                                                            • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                                                            • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                                                            Figure estimate

                                                                                                                                            30% of score weight measured. Score ≈40.2; missing-input range 23.2–48.2 points. API cost ≈$0.68 per mix task; range $0.13–$4.98.

                                                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                            • Briefcase analytical quality · score: 11.0 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 42 training models, 122 configurations. Held-out mean absolute error 12.52 points; benchmark-mean baseline 14.16.

                                                                                                                                              A higher or lower effort of the same model was measured, so the prediction is held at Gemini 3.5 Flash high’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (0.4 points). The held-out test does not establish accuracy here.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • Briefcase analytical quality · cost: $2.09 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 36 training models, 99 configurations. Held-out mean absolute error 1.00 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AA-Omniscience index · score: 46.2 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 34 training models, 88 configurations. Held-out mean absolute error 9.20 points; benchmark-mean baseline 10.03.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AA-Omniscience index · cost: $0.0096 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 30 training models, 68 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.34.

                                                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            Execute estimate

                                                                                                                                            0% of score weight measured. Score ≈21.0; missing-input range 1.8–29.2 points. API cost ≈$2.20 per mix task; range $0.35–$16.51.

                                                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                            • DeepSWE v1.1 · mini-swe-agent · score: 21.1 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1, on a curve that levels off near the top. 13 training models, 42 configurations. Held-out mean absolute error 8.45 points; benchmark-mean baseline 10.88.

                                                                                                                                              The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • DeepSWE v1.1 · mini-swe-agent · cost: $2.29 per scored attempt, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 13 training models, 40 configurations. Held-out mean absolute error 0.62 log dollars; benchmark-mean baseline 0.76.

                                                                                                                                              Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • Terminal-Bench 4.0 · AA harness · score: 9.9 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 42 training models, 121 configurations. Held-out mean absolute error 15.24 points; benchmark-mean baseline 16.92.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • Terminal-Bench 4.0 · AA harness · cost: $3.26 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 35 training models, 98 configurations. Held-out mean absolute error 0.93 log dollars; benchmark-mean baseline 1.28.

                                                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • GDPval-AA v2.1 · score: 34.8 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1, on a curve that levels off near the top. 42 training models, 125 configurations. Held-out mean absolute error 7.12 points; benchmark-mean baseline 8.48.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • GDPval-AA v2.1 · cost: $0.87 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 36 training models, 101 configurations. Held-out mean absolute error 0.92 log dollars; benchmark-mean baseline 1.27.

                                                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AutomationBench-AA partial credit · score: 37.8 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 42 training models, 122 configurations. Held-out mean absolute error 10.71 points; benchmark-mean baseline 11.31.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AutomationBench-AA partial credit · cost: $0.34 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: AA-LCR 1.1. 36 training models, 100 configurations. Held-out mean absolute error 0.46 log dollars; benchmark-mean baseline 1.18.

                                                                                                                                              Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            ≈40.18≈$0.6811≈21.04≈$2.1977
                                                                                                                                            Gemini 3.1 Pro Preview · high
                                                                                                                                            Evidence and eligibility

                                                                                                                                            Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                            Execute excluded from recommendations: Only 40% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                            • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                                                            • Figure · Briefcase analytical quality: Artificial Analysis has not published a model row for this configuration
                                                                                                                                            • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                                                                            • Figure · AA-LCR 1.1: Artificial Analysis has not published a model row for this configuration
                                                                                                                                            • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                            • Figure · AA-Omniscience index: Artificial Analysis has not published a model row for this configuration
                                                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                                                            • Execute · Terminal-Bench 4.0 · AA harness: Artificial Analysis has not published a model row for this configuration
                                                                                                                                            • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                                                            • Execute · GDPval-AA v2.1: Artificial Analysis has not published a model row for this configuration
                                                                                                                                            • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                                                            • Execute · AutomationBench-AA partial credit: Artificial Analysis has not published a model row for this configuration
                                                                                                                                            Figure estimate

                                                                                                                                            0% of score weight measured. Score ≈39.7; missing-input range 28.6–50.8 points. API cost ≈$0.64 per mix task; range $0.077–$5.42.

                                                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                            • Briefcase analytical quality · score: 4.7 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 11.70 points; benchmark-mean baseline 15.30.

                                                                                                                                              The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • Briefcase analytical quality · cost: $1.98 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.80 log dollars; benchmark-mean baseline 1.12.

                                                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AA-LCR 1.1 · score: 72.3 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 43 configurations. Held-out mean absolute error 2.44 points; benchmark-mean baseline 2.78.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AA-LCR 1.1 · cost: $0.16 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 41 configurations. Held-out mean absolute error 0.79 log dollars; benchmark-mean baseline 0.94.

                                                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AA-Omniscience index · score: 41.5 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 8 training models, 17 configurations. Held-out mean absolute error 7.44 points; benchmark-mean baseline 10.80.

                                                                                                                                              The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened; The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AA-Omniscience index · cost: $0.0089 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 8 training models, 17 configurations. Held-out mean absolute error 0.51 log dollars; benchmark-mean baseline 0.86.

                                                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            Execute estimate

                                                                                                                                            40% of score weight measured. Score ≈10.7; missing-input range 7.8–17.2 points. API cost ≈$1.97 per mix task; range $1.02–$8.52.

                                                                                                                                            Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                            • Terminal-Bench 4.0 · AA harness · score: 0.0 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 10.88 points; benchmark-mean baseline 14.45.

                                                                                                                                              The fitted curve passes the lowest measured score, so the prediction is held there. The held-out test does not establish accuracy here.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • Terminal-Bench 4.0 · AA harness · cost: $2.75 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.83 log dollars; benchmark-mean baseline 1.20.

                                                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • GDPval-AA v2.1 · score: 28.6 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent, on a curve that levels off near the top. 14 training models, 43 configurations. Held-out mean absolute error 4.48 points; benchmark-mean baseline 7.08.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • GDPval-AA v2.1 · cost: $0.85 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 40 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 1.13.

                                                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AutomationBench-AA partial credit · score: 21.2 converted points, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 14 training models, 40 configurations. Held-out mean absolute error 5.87 points; benchmark-mean baseline 10.18.

                                                                                                                                              Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            • AutomationBench-AA partial credit · cost: $0.41 per benchmark task, estimated.

                                                                                                                                              Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                              Predictor: DeepSWE v1.1 · mini-swe-agent. 13 training models, 39 configurations. Held-out mean absolute error 0.70 log dollars; benchmark-mean baseline 0.90.

                                                                                                                                              Missing measurement: Artificial Analysis has not published a model row for this configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                            ≈39.67≈$0.6445≈10.73≈$1.9659
                                                                                                                                            GPT-5.6 Terra · none
                                                                                                                                            Evidence and eligibility
                                                                                                                                              • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                              Execute estimate

                                                                                                                                              60% of score weight measured. Score ≈13.0; missing-input range 5.1–15.6 points. API cost ≈$0.37 per mix task; range $0.19–$1.06.

                                                                                                                                              Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                              • DeepSWE v1.1 · mini-swe-agent · score: 19.8 converted points, estimated.

                                                                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 13 training models, 35 configurations. Held-out mean absolute error 6.79 points; benchmark-mean baseline 11.08.

                                                                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                              • DeepSWE v1.1 · mini-swe-agent · cost: $0.63 per scored attempt, estimated.

                                                                                                                                                Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                Predictor: Terminal-Bench 4.0 · AA harness. 12 training models, 34 configurations. Held-out mean absolute error 0.56 log dollars; benchmark-mean baseline 0.73.

                                                                                                                                                Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                              36.35$0.1518≈13.02≈$0.3748
                                                                                                                                              DeepSeek V4 Pro 0813 · none
                                                                                                                                              Evidence and eligibility
                                                                                                                                                • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                                Execute estimate

                                                                                                                                                60% of score weight measured. Score ≈22.5; missing-input range 12.4–32.7 points. API cost ≈$1.83 per mix task; range $1.44–$2.51.

                                                                                                                                                Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                                • DeepSWE v1.1 · mini-swe-agent · score: 34.9 converted points, estimated.

                                                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                  Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                • DeepSWE v1.1 · mini-swe-agent · cost: $2.17 per scored attempt, estimated.

                                                                                                                                                  Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                  Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                                  Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                36.00$0.1468≈22.55≈$1.8277
                                                                                                                                                GPT-6 Luna · none
                                                                                                                                                Evidence and eligibility
                                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                                  Execute estimate

                                                                                                                                                  60% of score weight measured. Score ≈18.6; missing-input range 8.4–28.8 points. API cost ≈$0.12 per mix task; range $0.033–$0.75.

                                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 34.9 converted points, estimated.

                                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 14 training models, 40 configurations. Held-out mean absolute error 6.73 points; benchmark-mean baseline 11.75.

                                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.26 per scored attempt, estimated.

                                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 13 training models, 39 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.77.

                                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                  32.52$0.0091≈18.61≈$0.1209
                                                                                                                                                  GPT-5.6 Luna · none
                                                                                                                                                  Evidence and eligibility
                                                                                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                                  Figure estimate

                                                                                                                                                  60% of score weight measured. Score ≈27.1; missing-input range 19.3–34.8 points. API cost ≈$0.010 per mix task; range $0.0098–$0.024.

                                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                                  • AA-Omniscience index · score: 33.2 converted points, estimated.

                                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                    Predictor: AA-LCR 1.1. 33 training models, 87 configurations. Held-out mean absolute error 8.87 points; benchmark-mean baseline 9.78.

                                                                                                                                                    Missing measurement: No published measurement at this exact configuration. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                  • AA-Omniscience index · cost: $0.0014 per benchmark task, estimated.

                                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                    Predictor: AA-LCR 1.1. 29 training models, 67 configurations. Held-out mean absolute error 1.01 log dollars; benchmark-mean baseline 1.38.

                                                                                                                                                    Missing measurement: Token accounting is not published for this benchmark. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                  Execute estimate

                                                                                                                                                  60% of score weight measured. Score ≈4.8; missing-input range 3.2–4.8 points. API cost ≈$0.22 per mix task; range $0.023–$3.15.

                                                                                                                                                  Available ranges describe missing-input uncertainty, not all benchmark uncertainty or a confidence interval. Estimates set the displayed position; overlapping score ranges qualify the recommendation. Estimated costs may exceed a spending ceiling.

                                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · score: 3.8 converted points, estimated.

                                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness, on a curve that levels off near the top. 13 training models, 35 configurations. Held-out mean absolute error 5.77 points; benchmark-mean baseline 9.75.

                                                                                                                                                    A higher or lower effort of the same model was measured, so the prediction is held at Luna low’s measured score, loosened by the largest inversion measured on this benchmark between reasoning efforts (2.3 points). The held-out test does not establish accuracy here.

                                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                  • DeepSWE v1.1 · mini-swe-agent · cost: $0.52 per scored attempt, estimated.

                                                                                                                                                    Equal model weights; all efforts of each test model held out. The curve (straight or saturating) is whichever predicts held-out models better, held inside the measured ranges. Range uses the model-balanced 95th-percentile held-out error among the nearest third of measured configurations, which covered about 90% of held-out configurations, widened for extrapolation; not a confidence interval.

                                                                                                                                                    Predictor: Terminal-Bench 4.0 · AA harness. 12 training models, 34 configurations. Held-out mean absolute error 0.53 log dollars; benchmark-mean baseline 0.66.

                                                                                                                                                    The predictor value is outside every measured configuration, so the prediction is held at the edge of the measured range and its error allowance is widened. The held-out test does not establish accuracy here.

                                                                                                                                                    Missing measurement: No exact model and effort result in direct DeepSWE v1.1. The weekly refresh uses a compatible measured value as soon as the source supplies one.

                                                                                                                                                  ≈27.08≈$0.0103≈4.77≈$0.2169
                                                                                                                                                  GPT-5.5 Pro · xhigh
                                                                                                                                                  Evidence and eligibility

                                                                                                                                                  Figure excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                                  Execute excluded from recommendations: Only 0% of score weight measured; at least 50% required for a recommendation.

                                                                                                                                                  • Figure · Briefcase analytical quality: No published measurement at this exact configuration
                                                                                                                                                  • Figure · Briefcase analytical quality: Token accounting is not published for this benchmark
                                                                                                                                                  • Figure · AA-LCR 1.1: No published measurement at this exact configuration
                                                                                                                                                  • Figure · AA-LCR 1.1: Token accounting is not published for this benchmark
                                                                                                                                                  • Figure · AA-Omniscience index: No published measurement at this exact configuration
                                                                                                                                                  • Figure · AA-Omniscience index: Token accounting is not published for this benchmark
                                                                                                                                                  • Execute · DeepSWE v1.1 · mini-swe-agent: No exact model and effort result in direct DeepSWE v1.1
                                                                                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: No published measurement at this exact configuration
                                                                                                                                                  • Execute · Terminal-Bench 4.0 · AA harness: Token accounting is not published for this benchmark
                                                                                                                                                  • Execute · GDPval-AA v2.1: No published measurement at this exact configuration
                                                                                                                                                  • Execute · GDPval-AA v2.1: Token accounting is not published for this benchmark
                                                                                                                                                  • Execute · AutomationBench-AA partial credit: No published measurement at this exact configuration
                                                                                                                                                  • Execute · AutomationBench-AA partial credit: Token accounting is not published for this benchmark
                                                                                                                                                  UnscoredUnpricedUnscoredUnpriced
                                                                                                                                                  Data: November 11, 2026 · Artificial Analysis, Datacurve · supporting: Cognition, ZapierPrices here compare setups against each other. Your actual bill will differ.