HumanEval
18 models ranked
HumanEval is OpenAI’s benchmark for code generation: 164 hand-written programming problems that test whether a model can produce a correct Python function from a docstring. Scores are reported as pass@1, the share of problems solved on the first attempt. It is one of the oldest and most widely cited coding benchmarks, even if real-world programming quickly outgrew it.
Scores from BenchLM
| # | Model | ||
|---|---|---|---|
| 1 | GPT-5 | OpenAI | 100 |
| 2 | Claude 4 Opus | Anthropic | 100 |
| 3 | Gemini 2.5 Pro | 100 | |
| 4 | Cursor Tab | Anysphere | 100 |
| 5 | Llama 4 405B | Meta | 99 |
| 6 | Claude Code | Anthropic | 99 |
| 7 | Grok 3 | xAI | 98 |
| 8 | Qwen3 235B | Alibaba | 97 |
| 9 | DeepSeek V3 | DeepSeek | 96 |
| 10 | GitHub Copilot | GitHub | 96 |
| 11 | Mistral Large 3 | Mistral | 95 |
| 12 | Claude 4 Sonnet | Anthropic | 94 |
| 13 | Aider | Aider | 93 |
| 14 | Command R+ | Cohere | 92 |
| 15 | Codeium Windsurf | Codeium | 92 |
| 16 | Gemini 2.5 Flash | 91 | |
| 17 | GPT-4o mini | OpenAI | 89 |
| 18 | Continue Dev | Continue | 88 |
AIRank aggregates these results from the public sources listed on each page. We do not run our own evaluations, and we normalize raw results to a 0–100 scale so tables are comparable.
FAQ
Frequently asked questions
What is HumanEval?
HumanEval is OpenAI’s benchmark for code generation: 164 hand-written programming problems that test whether a model can produce a correct Python function from a docstring. Scores are reported as pass@1, the share of problems solved on the first attempt. It is one of the oldest and most widely cited coding benchmarks, even if real-world programming quickly outgrew it. The HumanEval results shown here come from the public scores published by the benchmark’s own team and re-aggregated through sources such as BenchLM.
How are benchmark scores computed?
Each benchmark reports its results on its own native scale — percentages, pass@1 rates or Elo points. We normalize every benchmark to a shared 0–100 scale using min-max scaling over the models we track, so tables are comparable across benchmarks. The underlying numbers always come from the public results published by the benchmark maintainers, never from proprietary or self-run evaluations.
Why do the same models rank differently across benchmarks?
Benchmarks measure different skills. GPQA tests graduate-level science knowledge, HumanEval and SWE-Bench measure coding ability, and MATH covers step-by-step problem solving. A model can be excellent at code and average at science — that is exactly why we keep a separate leaderboard for each benchmark instead of merging everything into one score.