Benchmark · Coding
Terminal-Bench 4.0
% tasks resolved (%) Higher is better Benchmark homepage
What it measures
Terminal-Bench measures how well AI agents complete tasks using a terminal (command-line environment). It is hosted by Stanford, Harbor and the Laude Institute and runs on the Harbor framework. Version 4.0 recalibrated task resources (time, CPU, memory), fixed 19 tasks, removed 8 saturated tasks and set a flat 8-hour agent timeout for every task. The official leaderboard reports accuracy with a confidence margin for each combination of agent harness, model and reasoning effort.
Caveats:
Scores depend heavily on the agent harness (for example Claude Code, Codex, Grok Build or mini-SWE-agent) and on the reasoning effort, so the same model appears several times on the leaderboard. Leaderboard rows are often submitted by the organisation that builds the agent. Providers' own runs can differ from the leaderboard: Anthropic reports 66.4% for Claude Opus 5.5 at max effort, while the leaderboard lists 64.8% ± 3.1%. Versions 2.0, 2.1 and 3.0 and Artificial Analysis' "Terminal-Bench Hard" use different task sets and are not comparable with 4.0. Margins are about ±3 points.
Results
| Model | Score | Setting | Reported by | Measured | Verified | Source |
|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Anthropic | 70.6 % | max effort; Claude Code (--bare), 5 trials/task | Self-reported | — | 2026-10-12 | Claude Sonnet 5.5 System Card (Anthropic, 2026-09-28) |
| Claude Opus 5.5 Anthropic | 66.4 % | xhigh effort; Claude Code (--bare), 5 trials/task | Self-reported | — | 2026-10-12 | Claude Opus 5.5 System Card (Anthropic, 2026-09-22) |
| Claude Opus 5.5 Anthropic | 64.8 % | max effort; Claude Code (--bare), 5 trials/task | Self-reported | — | 2026-10-12 | Claude Opus 5.5 System Card (Anthropic, 2026-09-22) |
| Claude Opus 5.5 Anthropic | 64.8 % | max effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| Claude Sonnet 5.5 Anthropic | 61.8 % | max effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| GPT-6 Astra OpenAI | 58.2 % | max effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| GPT-6.1 Sol OpenAI | 58.2 % | max effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| Claude Fable 5.1 Anthropic | 57.9 % | max effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| Claude Fable 5.1 Anthropic | 57.9 % | xhigh effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| GPT-6 Astra OpenAI | 57.9 % | high effort | Self-reported | — | 2026-10-25 | GPT-6 Astra: A new generation of intelligence | OpenAI |
| GPT-6 Astra OpenAI | 57.9 % | xhigh effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| GPT-6 Astra OpenAI | 57.9 % | high effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| GPT-6 Astra OpenAI | 57.6 % | xhigh effort | Self-reported | — | 2026-10-25 | GPT-6 Astra: A new generation of intelligence | OpenAI |
| Gemini 4 Argon Google | 57.4 % | high effort; harness not stated (self-computed by Google) | Self-reported | — | 2026-10-25 | Gemini 4 Argon model card - Google DeepMind |
| Muse Wonder Full Meta | 57.1 % | Terminal-Bench 4.0; max effort; result sourced by Meta from Artificial Analysis (Artificial Analysis harness) | Self-reported | — | 2026-10-25 | Muse Wonder Model Card (PDF) |
| GPT-6 Astra OpenAI | 56.7 % | max effort | Self-reported | — | 2026-10-25 | GPT-6 Astra: A new generation of intelligence | OpenAI |
| Claude Fable 5.1 Anthropic | 55.8 % | max effort; Claude Code (--bare), 15 trials/task | Self-reported | — | 2026-10-12 | Claude Fable 5.1 & Claude Mythos 5.1 System Card (Anthropic, 2026-09-01) |
| Claude Fable 5.1 Anthropic | 54.5 % | high effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| GPT-6 Astra OpenAI | 54.2 % | medium effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| Claude Fable 5.1 Anthropic | 53.9 % | medium effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| GPT-6 Astra OpenAI | 53.9 % | medium effort | Self-reported | — | 2026-10-25 | GPT-6 Astra: A new generation of intelligence | OpenAI |
| Muse Wonder Meta | 52 % | Terminal-Bench 4.0; max effort; result sourced by Meta from Artificial Analysis (Artificial Analysis harness) | Self-reported | — | 2026-10-25 | Muse Wonder Model Card (PDF) |
| GPT-6 Astra OpenAI | 50.6 % | low effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| GPT-6 Astra OpenAI | 49.7 % | low effort | Self-reported | — | 2026-10-25 | GPT-6 Astra: A new generation of intelligence | OpenAI |
| GPT-6 Sol OpenAI | 49.4 % | max effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| Claude Fable 5.1 Anthropic | 43.3 % | low effort; Claude Code harness | Third party | — | 2026-10-12 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| DeepSeek-V4.1-Flash DeepSeek | 42.3 % | snapshot DeepSeek-V4.1-Flash-1017; effort not stated; Claude Code harness | Self-reported | — | 2026-10-25 | deepseek-ai/DeepSeek-V4.1-Flash-1017 model card (Hugging Face) |
| Claude Haiku 5.5 Anthropic | 39.2 % | max effort; Claude Code (--bare), 10 trials/task | Self-reported | — | 2026-10-12 | Claude Haiku 5.5 System Card (Anthropic, 2026-10-07) |
| Grok 4.7 xAI | 37.6 % | xhigh effort; Grok Build harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| Grok 4.7 xAI | 37.6 % | xhigh effort | Self-reported | — | 2026-10-25 | Introducing Grok 4.7 |
| DeepSeek-V4.1-Flash DeepSeek | 31.2 % | Pass@1; max effort (reasoning_effort=100); DeepSeek Harness (Minimal mode), 1M-token context | Self-reported | — | 2026-10-25 | deepseek-ai/DeepSeek-V4.1-Flash model card (Hugging Face) |
| Mistral Large 4 Mistral AI | 28.3 % | Terminal-Bench 4.0, 'Mistral Large 4 Preview'; effort not stated; Artificial Analysis harness (run privately by AA before the harness' public launch) | Self-reported | — | 2026-10-25 | Introducing Mistral Large 4 | Mistral |
| Qwen3.8-Max Alibaba Cloud (Qwen) | 27 % | snapshot qwen3.8-max-0902; max effort; Claude Code harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| Grok 4.6 xAI | 20.3 % | high effort; Grok Build harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| Grok 4.6 xAI | 20.3 % | high effort | Self-reported | — | 2026-10-25 | Model Card: Grok 4.7 (Revision 2026-09-21) |
| Gemini 3.8 Flash Google | 19.1 % | high effort; mini-SWE-agent harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| GPT-6 Luna OpenAI | 16.4 % | max effort; Codex harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| Muse Spark 1.3 Meta | 14.5 % | Terminal-Bench 4.0; xhigh effort; Muse Code harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (Harbor Hub) |
| DeepSeek-V4-Pro DeepSeek | 12.4 % | Pass@1; max effort; harness not stated | Self-reported | — | 2026-10-25 | deepseek-ai/DeepSeek-V4.1-Flash model card (Hugging Face) |
| Grok 4.5 xAI | 12.4 % | high effort; Grok Build harness | Third party | — | 2026-10-25 | Terminal-Bench 4.0 leaderboard (tbench.ai) |
| Grok 4.5 xAI | 12.4 % | high effort | Self-reported | — | 2026-10-25 | Model Card: Grok 4.7 (Revision 2026-09-21) |
Sorted by score, without rank numbers: each result was measured under its own setting (effort level, harness, tools, test version), as stated by the source. Scores are only directly comparable between rows with the same setting. Self-reported results come from the provider and have not been reproduced independently.
Sources
- Terminal-Bench 4.0
- Terminal-Bench
- Claude Sonnet 5.5 System Card (Anthropic, 2026-09-28)
- Claude Opus 5.5 System Card (Anthropic, 2026-09-22)
- Terminal-Bench 4.0 leaderboard (tbench.ai)
- Terminal-Bench 4.0 leaderboard (Harbor Hub)
- GPT-6 Astra: A new generation of intelligence | OpenAI
- Gemini 4 Argon model card - Google DeepMind
- Muse Wonder Model Card (PDF)
- Claude Fable 5.1 & Claude Mythos 5.1 System Card (Anthropic, 2026-09-01)
- deepseek-ai/DeepSeek-V4.1-Flash-1017 model card (Hugging Face)
- Claude Haiku 5.5 System Card (Anthropic, 2026-10-07)
- Introducing Grok 4.7
- deepseek-ai/DeepSeek-V4.1-Flash model card (Hugging Face)
- Introducing Mistral Large 4 | Mistral
- Model Card: Grok 4.7 (Revision 2026-09-21)