Tool · Updated 30 September 2026

AI model benchmarks for Insights and Strategy

  • Opus 5.5 leads 7 of 9 benchmarks, with a 60% lower token price than GPT-6 Astra.
  • For coding with Opus 5.5, stop at medium effort. At max effort, Opus 5.5 passes 10% more tasks than at medium, for 4.6 times the cost per task.
  • GPT-6 Astra's tokens cost 2.5 times as much as Opus 5.5's. Its only clear lead is reasoning and science: on Terminal-Bench-Science it passes 10% more tasks than Opus 5.5.

Best choice by task

Coding and data scripts

Opus 5.5medium effort

53% on CursorBench 4.0$2.91 a task

vs Sonnet 5.5 at max
6% more tasks, 3.3 times the cost
vs Opus 5.5 at high
7% more tasks, 36% more cost

At max effort, Opus 5.5 passes 10% more tasks than at medium, for 4.6 times the cost per task.

Cost data from the benchmark owner

Reports and client deliverables

Opus 5.5

1846 Elo on GDPval-AA 2.1No cost per task published

vs Fable 5.1
rated 111 points lower, 2.5 times the token price
vs GPT-5.6 Sol
rated 258 points lower, the same token price

GPT-6 Astra is rated 304 points lower on this benchmark.

Matched by a third party

Reporting and workflow automation

Sonnet 5.5xhigh effort

37% on AutomationBench$0.48 a task

vs Opus 5.5 at max
15% more tasks, 3 times the cost
vs Sonnet 5.5 at max
22% more tasks, 2.4 times the cost

Zapier reran steps the Anthropic models refused on Anthropic's fallback model. Anthropic's launch table, run without fallback, gives Opus 5.5 40%.

Cost data from the benchmark owner

Research analysis

GPT-6 Astra

65% on Terminal-Bench-Science 0.1No cost per task published

vs Opus 5.5
9% fewer tasks, 60% lower token price

For expert questions answered with tools, Opus 5.5 answers 3% more questions than Fable 5.1 on Humanity's Last Exam.

Vendor-reported

Working in tools without an API

Opus 5.5

82% on OSWorld 2.0No cost per task published

vs Fable 5.1
1% fewer tasks, 2.5 times the token price

OpenAI's OSWorld 2.0 figures come from its launch, on a scoring mode that is not confirmed, and the two Chartography sources name different leaders.

Vendor-reported

Top models comparison by key benchmarks

Each bar shows how many more or fewer tasks the first model passes than the second, as a share of the second model's score. A gap inside the margin of error counts as a tie.

Launch table, 22 September 2026

Opus 5.5 ahead on 3 GPT-6 Astra ahead on 1 2 ties 3 without a comparison

Opus 5.5 against GPT-6 Astra: 60% lower token price; 17% less cost per task at max effort on AutomationBench.

Agentic coding

Terminal-Bench 4.0 15% more tasks66.4 vs 57.9
FrontierCode 1.1 Main Tie, 2% more tasks54.4 vs 53.3
CursorBench 4.0 No published score for GPT-6 Astra

Knowledge work

GDPval-AA 2.1 rated 304 points higher1846 vs 1542

Business workflows

AutomationBench 1.0.6 Tie, 3% fewer tasks40.0 vs 41.4

Reasoning and science

Humanity's Last Exam 18% more tasks67.7 vs 57.2
Terminal-Bench-Science 0.1 9% fewer tasks58.7 vs 64.6

Computer use and charts

OSWorld 2.0 No comparison on the same setting: GPT-6 Astra has a score only from another source
Chartography No comparison on the same setting: GPT-6 Astra has a score only from another source

Bars show the difference in tasks passed, up to 40% either side. The GDPval-AA row shows rating points, up to 400 either side.

Pass rate and cost per task

Price per token is an unfair comparison, because models use different numbers of tokens to complete a task. GPT-6 Astra's tokens cost 2.5 times as much as Opus 5.5's, yet at max effort on AutomationBench it costs 20% more per task.

Each chart plots the share of tasks a model completed correctly against what its provider charged per task, as published by the benchmark owner.

How we set model and effort in daily work is covered in our Claude Code workflow and Codex power-user workflow guides. Our guide to agentic RAG weighs capability against running cost for retrieval systems.

CursorBench 4.0

Best score for the cost A better option exists at the same price or less
CursorBench 4.0: pass rate against cost per task Opus 5.5 at low effort: 43.7% for $1.17 a task. Opus 5.5 at medium effort: 52.5% for $2.91 a task. Opus 5.5 at high effort: 56.0% for $3.97 a task. Opus 5.5 at xhigh effort: 56.0% for $6.98 a task. Opus 5.5 at max effort: 57.8% for $13.43 a task. Fable 5.1 at low effort: 45.1% for $5.44 a task. Fable 5.1 at medium effort: 46.8% for $7.05 a task. Fable 5.1 at high effort: 49.2% for $9.08 a task. Fable 5.1 at xhigh effort: 51.6% for $13.01 a task. Fable 5.1 at max effort: 51.8% for $17.28 a task. Opus 5 at low effort: 40.7% for $4.87 a task. Opus 5 at medium effort: 43.3% for $6.94 a task. Opus 5 at high effort: 44.7% for $9.00 a task. Opus 5 at xhigh effort: 46.1% for $11.43 a task. Opus 5 at max effort: 46.6% for $11.95 a task. GPT-5.6 Sol at low effort: 24.6% for $0.87 a task. GPT-5.6 Sol at medium effort: 31.1% for $1.77 a task. GPT-5.6 Sol at high effort: 35.7% for $2.85 a task. GPT-5.6 Sol at xhigh effort: 37.7% for $4.40 a task. GPT-5.6 Sol at max effort: 41.7% for $8.23 a task. Sonnet 5.5 at low effort: 35.8% for $0.50 a task. Sonnet 5.5 at medium effort: 39.2% for $0.70 a task. Sonnet 5.5 at high effort: 47.8% for $1.67 a task. Sonnet 5.5 at xhigh effort: 53.1% for $3.88 a task. Sonnet 5.5 at max effort: 55.5% for $9.67 a task. 20% 25% 30% 35% 40% 45% 50% 55% 60% $0.50 $1 $2 $3 $5 $10 $20 Cost per task (US dollars, log scale) Pass rate low medium high xhigh max Opus 5.5 Fable 5.1 Opus 5 GPT-5.6 Sol Sonnet 5.5

Cursor lists no GPT-6 Astra or GPT-6.1 Sol row. Source: Cursor, CursorBench 4.0.

Gain from each step up in effort

Opus 5.5

low → medium 20% more tasks 2.5 times the cost
medium → high 7% more tasks 36% more cost
high → xhigh no gain 76% more cost
xhigh → max 3% more tasks 92% more cost

Fable 5.1

low → medium 4% more tasks 30% more cost
medium → high 5% more tasks 29% more cost
high → xhigh 5% more tasks 43% more cost
xhigh → max under 1% more tasks 33% more cost

Opus 5

low → medium 6% more tasks 43% more cost
medium → high 3% more tasks 30% more cost
high → xhigh 3% more tasks 27% more cost
xhigh → max 1% more tasks 5% more cost

GPT-5.6 Sol

low → medium 26% more tasks 2 times the cost
medium → high 15% more tasks 61% more cost
high → xhigh 6% more tasks 54% more cost
xhigh → max 11% more tasks 87% more cost

Sonnet 5.5

low → medium 9% more tasks 40% more cost
medium → high 22% more tasks 2.4 times the cost
high → xhigh 11% more tasks 2.3 times the cost
xhigh → max 5% more tasks 2.5 times the cost
Show all 25 options as a table
Model and effortPass rateCost per task
Opus 5.5 at max57.8%$13.43
Opus 5.5 at high56.0%$3.97
Opus 5.5 at xhigh56.0%$6.98
Sonnet 5.5 at max55.5%$9.67
Sonnet 5.5 at xhigh53.1%$3.88
Opus 5.5 at medium sweet spot52.5%$2.91
Fable 5.1 at max51.8%$17.28
Fable 5.1 at xhigh51.6%$13.01
Fable 5.1 at high49.2%$9.08
Sonnet 5.5 at high47.8%$1.67
Fable 5.1 at medium46.8%$7.05
Opus 5 at max46.6%$11.95
Opus 5 at xhigh46.1%$11.43
Fable 5.1 at low45.1%$5.44
Opus 5 at high44.7%$9.00
Opus 5.5 at low43.7%$1.17
Opus 5 at medium43.3%$6.94
GPT-5.6 Sol at max41.7%$8.23
Opus 5 at low40.7%$4.87
Sonnet 5.5 at medium39.2%$0.70
GPT-5.6 Sol at xhigh37.7%$4.40
Sonnet 5.5 at low35.8%$0.50
GPT-5.6 Sol at high35.7%$2.85
GPT-5.6 Sol at medium31.1%$1.77
GPT-5.6 Sol at low24.6%$0.87

AutomationBench

Best score for the cost A better option exists at the same price or less
AutomationBench: pass rate against cost per task Opus 5.5 at low effort: 24.2% for $0.51 a task. Opus 5.5 at medium effort: 29.5% for $0.65 a task. Opus 5.5 at high effort: 33.0% for $0.71 a task. Opus 5.5 at xhigh effort: 35.8% for $0.89 a task. Opus 5.5 at max effort: 42.5% for $1.44 a task. GPT-6 Astra at low effort: 30.3% for $1.08 a task. GPT-6 Astra at medium effort: 34.1% for $1.27 a task. GPT-6 Astra at high effort: 37.1% for $1.44 a task. GPT-6 Astra at xhigh effort: 39.0% for $1.50 a task. GPT-6 Astra at max effort: 41.4% for $1.73 a task. Fable 5.1 at max effort: 31.4% for $2.45 a task. Sonnet 5.5 at low effort: 18.3% for $0.28 a task. Sonnet 5.5 at medium effort: 23.3% for $0.31 a task. Sonnet 5.5 at high effort: 29.4% for $0.36 a task. Sonnet 5.5 at xhigh effort: 36.8% for $0.48 a task. Sonnet 5.5 at max effort: 44.8% for $1.14 a task. 15% 20% 25% 30% 35% 40% 45% 50% $0.50 $1 $2 $3 Cost per task (US dollars, log scale) Pass rate low medium high xhigh max Opus 5.5 low medium high xhigh max GPT-6 Astra max Fable 5.1 low medium high xhigh max Sonnet 5.5

Zapier's Opus 5.5 and Sonnet 5.5 rows use default fallbacks: a refused step is rerun on Anthropic's fallback model. For Fable 5.1, Opus 5 completed steps Fable's safety classifier refused, about 40% of tasks (260 of 657), and the cost covers Fable 5.1 only. Zapier does not list GPT-6.1 Sol yet. Source: Zapier, AutomationBench v1.0.6.

Gain from each step up in effort

Opus 5.5

low → medium 22% more tasks 27% more cost
medium → high 12% more tasks 9% more cost
high → xhigh 8% more tasks 25% more cost
xhigh → max 19% more tasks 62% more cost

GPT-6 Astra

low → medium 13% more tasks 18% more cost
medium → high 9% more tasks 13% more cost
high → xhigh 5% more tasks 4% more cost
xhigh → max 6% more tasks 15% more cost

Sonnet 5.5

low → medium 28% more tasks 11% more cost
medium → high 26% more tasks 16% more cost
high → xhigh 25% more tasks 33% more cost
xhigh → max 22% more tasks 2.4 times the cost
Show all 16 options as a table
Model and effortPass rateCost per task
Sonnet 5.5 at max44.8%$1.14
Opus 5.5 at max42.5%$1.44
GPT-6 Astra at max41.4%$1.73
GPT-6 Astra at xhigh39.0%$1.50
GPT-6 Astra at high37.1%$1.44
Sonnet 5.5 at xhigh sweet spot36.8%$0.48
Opus 5.5 at xhigh35.8%$0.89
GPT-6 Astra at medium34.1%$1.27
Opus 5.5 at high33.0%$0.71
Fable 5.1 at max31.4%$2.45
GPT-6 Astra at low30.3%$1.08
Opus 5.5 at medium29.5%$0.65
Sonnet 5.5 at high29.4%$0.36
Opus 5.5 at low24.2%$0.51
Sonnet 5.5 at medium23.3%$0.31
Sonnet 5.5 at low18.3%$0.28

All scores and sources

Anthropic's launch table of 22 September 2026. Each score shows its gap to the leader on that row. Scores for OpenAI models in this view are Anthropic's figures.

Benchmark Opus 5.5 Anthropic Sonnet 5.5 Anthropic Fable 5.1 Anthropic Opus 5 Anthropic GPT-6 Astra OpenAI GPT-6.1 Sol OpenAI GPT-5.6 Sol OpenAI Lead
Agentic coding
Terminal-Bench 4.0 64.1 Vals AI, other setting 55.0 Vals AI, other setting Opus 5.5 +8.5 points, clear
FrontierCode 1.1 Main 52.1 Anthropic, other setting No score found Opus 5.5 +1.1 points, narrow
CursorBench 4.0 55.5 Cursor, other setting No score found No score found Opus 5.5 +6.0 points, clear
Knowledge work
GDPval-AA 2.1 · Elo 1844 Artificial Analysis, other setting 1575 Artificial Analysis, other setting Opus 5.5 +111 Elo, clear
Business workflows
AutomationBench 1.0.6 44.8 Zapier, other setting 36.1 OpenAI, other setting GPT-6 Astra +1.4 points, narrow
Reasoning and science
Humanity's Last Exam with tools 55.0 Artificial Analysis, other setting 52.9 Artificial Analysis, other setting No score found Opus 5.5 +2.1 points, clear
Terminal-Bench-Science 0.1 No score found 57.0 OpenAI, other setting GPT-6 Astra +5.9 points, clear
Computer use and charts
OSWorld 2.0 No score found 72.6 OpenAI, other setting 71.4 OpenAI, other setting 65.7 OpenAI, other setting Opus 5.5 +1.1 points, narrow
Chartography with tools No score found 71.0 Surge AI, other setting No score found 45.0 Surge AI, other setting Opus 5.5 +0.6 points, narrow
  • An independent leaderboard reports the same score.
  • Another source reports a different score, listed under conflicting figures.
  • Reported by Anthropic. We could not confirm OpenAI's own figure.
  • Not in Anthropic's table; another source publishes this score on a different setting, so it is not compared.

Benchmark definitions

Terminal-Bench 4.0
Whether an agent can finish real jobs in a command-line terminal: set up software, fix builds, process data, run a server. Share of tasks fully resolved, averaged over three runs.
FrontierCode 1.1 Main
Whether a code change written by the agent would be accepted into a real software project. Graded with the project's tests, maintainer rubrics and automated verifiers.
CursorBench 4.0
Coding requests taken from real sessions in the Cursor editor: vague briefs, several files, and some long tasks. Share of tasks solved.
GDPval-AA 2.1
Everyday professional deliverables such as reports, spreadsheets and plans, drawn from 44 occupations. An AI judge compares outputs blind, in pairs, and the results are scored as an Elo rating. Higher means preferred more often.
AutomationBench 1.0.6
Multi-step business workflows across connected apps: CRM updates, email, spreadsheets, ticketing. A task counts only when every check on the final state of the connected systems passes.
Humanity's Last Exam with tools
Expert-written questions across more than a hundred subjects, written to be hard for frontier models. Share of questions answered correctly.
Terminal-Bench-Science 0.1
Research tasks run end to end in a terminal: analysing data, running simulations, reproducing results. Share of tasks resolved.
OSWorld 2.0
Using a real desktop computer through the screen, mouse and keyboard to finish long tasks across ordinary apps. Anthropic reports a partial score, which credits progress on unfinished tasks, and a strict score for finished tasks only. This page uses partial.
Chartography with tools
Reading professional charts correctly and answering questions about them. Mean pass rate over 20 trials.

Conflicting figures

Scores from other sources

  • Sonnet 5.5, Terminal-Bench 4.0: not in the launch table; 64.1 in Vals AI, Terminal-Bench 4.0 leaderboard (mini-swe-agent setup, mean of three runs).
  • GPT-6.1 Sol, Terminal-Bench 4.0: not in the launch table; 55.0 in Vals AI, Terminal-Bench 4.0 leaderboard (mini-swe-agent setup, mean of three runs).
  • Sonnet 5.5, FrontierCode 1.1 Main: not in the launch table; 52.1 in Anthropic, Claude Sonnet 5.5 launch page (Sonnet 5.5 launch page, xhigh effort).
  • Sonnet 5.5, CursorBench 4.0: not in the launch table; 55.5 in Cursor, CursorBench 4.0 (Cursor's leaderboard, max effort).
  • Sonnet 5.5, GDPval-AA 2.1: not in the launch table; 1844 in Artificial Analysis, GDPval-AA leaderboard (Max effort).
  • GPT-6.1 Sol, GDPval-AA 2.1: not in the launch table; 1575 in Artificial Analysis, GDPval-AA leaderboard (Max effort).
  • Sonnet 5.5, AutomationBench 1.0.6: not in the launch table; 44.8 in Zapier, AutomationBench v1.0.6 (Max effort, default fallbacks: refused steps rerun on Anthropic's fallback model).
  • GPT-6.1 Sol, AutomationBench 1.0.6: not in the launch table; 36.1 in OpenAI, GPT-6.1 Sol launch page (OpenAI's launch chart, max effort).
  • Sonnet 5.5, Humanity's Last Exam: not in the launch table; 55.0 in Artificial Analysis, Humanity's Last Exam leaderboard (Text-only questions, tool use not stated).
  • GPT-6.1 Sol, Humanity's Last Exam: not in the launch table; 52.9 in Artificial Analysis, Humanity's Last Exam leaderboard (Text-only questions, tool use not stated).
  • GPT-6.1 Sol, Terminal-Bench-Science 0.1: not in the launch table; 57.0 in OpenAI, GPT-6.1 Sol launch page (OpenAI's launch chart, max effort).
  • GPT-6 Astra, OSWorld 2.0: not in the launch table; 72.6 in OpenAI's launch table as reported by The Decoder (OpenAI's launch figure, reported in the press; scoring mode not confirmed).
  • GPT-6.1 Sol, OSWorld 2.0: not in the launch table; 71.4 in OpenAI, GPT-6.1 Sol launch page (OpenAI's launch chart, max effort, partial reward on the offline set).
  • GPT-5.6 Sol, OSWorld 2.0: not in the launch table; 65.7 in OpenAI's launch table as reported by The Decoder (OpenAI's launch figure, reported in the press; scoring mode not confirmed).
  • GPT-6 Astra, Chartography: not in the launch table; 71.0 in Surge AI, Chartography leaderboard (Max effort, Surge's own protocol, tool use not stated).
  • GPT-5.6 Sol, Chartography: not in the launch table; 45.0 in Surge AI, Chartography leaderboard (Max effort, Surge's own protocol, tool use not stated).

Third-party checks

  • GDPval-AA 2.1: Artificial Analysis reports the same 5 scores as the launch table.

Sources

  1. Anthropic, Claude Opus 5.5 launch page vendor
  2. LLM Stats, FrontierCode 1.1 (self-reported figures) aggregator
  3. Anthropic, Claude Fable 5.1 launch page vendor
  4. Vals AI, Terminal-Bench 4.0 leaderboard independent
  5. Artificial Analysis, GDPval-AA leaderboard independent
  6. Zapier, AutomationBench v1.0.6 independent
  7. Artificial Analysis, Humanity's Last Exam leaderboard independent
  8. Terminal-Bench, Terminal-Bench-Science 0.1 results independent
  9. Surge AI, Chartography leaderboard independent
  10. Cursor, CursorBench 4.0 independent
  11. OpenAI's launch table as reported by The Decoder secondary
  12. OpenAI, GPT-6.1 Sol launch page vendor
  13. Anthropic, Claude Sonnet 5.5 launch page vendor