SWE-Bench
18 models ranked
SWE-Bench is the agentic coding benchmark: it gives a model a real GitHub issue from an open-source repository and asks it to produce a patch that passes the project’s tests. With more than 2,000 tasks, it measures whether an AI can navigate unfamiliar codebases, understand bug reports and land fixes that actually merge. It is the closest public proxy for a hired software engineer.
Scores from BenchLM
| # | Model | ||
|---|---|---|---|
| 1 | GPT-5 | OpenAI | 93 |
| 2 | Claude 4 Opus | Anthropic | 92 |
| 3 | Gemini 2.5 Pro | 91 | |
| 4 | Cursor Tab | Anysphere | 90 |
| 5 | Llama 4 405B | Meta | 89 |
| 6 | Claude Code | Anthropic | 89 |
| 7 | Grok 3 | xAI | 88 |
| 8 | Qwen3 235B | Alibaba | 87 |
| 9 | DeepSeek V3 | DeepSeek | 86 |
| 10 | GitHub Copilot | GitHub | 86 |
| 11 | Mistral Large 3 | Mistral | 85 |
| 12 | Claude 4 Sonnet | Anthropic | 84 |
| 13 | Aider | Aider | 83 |
| 14 | Command R+ | Cohere | 82 |
| 15 | Codeium Windsurf | Codeium | 82 |
| 16 | Gemini 2.5 Flash | 81 | |
| 17 | GPT-4o mini | OpenAI | 79 |
| 18 | Continue Dev | Continue | 78 |
AIRank aggregates these results from the public sources listed on each page. We do not run our own evaluations, and we normalize raw results to a 0–100 scale so tables are comparable.
FAQ
Frequently asked questions
What is SWE-Bench?
SWE-Bench is the agentic coding benchmark: it gives a model a real GitHub issue from an open-source repository and asks it to produce a patch that passes the project’s tests. With more than 2,000 tasks, it measures whether an AI can navigate unfamiliar codebases, understand bug reports and land fixes that actually merge. It is the closest public proxy for a hired software engineer. The SWE-Bench results shown here come from the public scores published by the benchmark’s own team and re-aggregated through sources such as BenchLM.
How are benchmark scores computed?
Each benchmark reports its results on its own native scale — percentages, pass@1 rates or Elo points. We normalize every benchmark to a shared 0–100 scale using min-max scaling over the models we track, so tables are comparable across benchmarks. The underlying numbers always come from the public results published by the benchmark maintainers, never from proprietary or self-run evaluations.
Why do the same models rank differently across benchmarks?
Benchmarks measure different skills. GPQA tests graduate-level science knowledge, HumanEval and SWE-Bench measure coding ability, and MATH covers step-by-step problem solving. A model can be excellent at code and average at science — that is exactly why we keep a separate leaderboard for each benchmark instead of merging everything into one score.