Software Engineering on SWE-bench Verified (Pass@1 and Time Metrics)
72Pass@1
GPT-5 mini (2025-08-07)
Updated 18d ago
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
2026.07 | 72 | - | - | - | - | - | - | |
| 66.8 | - | - | - | - | - | - | ||
| 66.5 | - | - | - | - | - | - | ||
| 62.5 | - | - | - | - | - | - | ||
2026.07 | 62 | - | - | - | - | - | - | |
2026.07 | 59.8 | - | - | - | - | - | - | |
| 58.3 | - | - | - | - | - | - | ||
2026.07 | 58 | - | - | - | - | - | - | |
2026.07 | 57.8 | - | - | - | - | - | - | |
| 57.3 | - | - | - | - | - | - | ||
| 56 | - | - | - | - | - | - | ||
2026.07 | 51.9 | - | - | - | - | - | - | |
2026.07 | 50.5 | - | - | - | - | - | - | |
| 50 | - | - | - | - | - | - | ||
| 48 | - | - | - | - | - | - | ||
2026.07 | 47.5 | - | - | - | - | - | - | |
| 45.2 | - | - | - | - | - | - | ||
| 43.7 | - | - | - | - | - | - | ||
2026.05 | 42.2 | 172.9 | 168.1 | 290 | 282 | 2,146 | 39.8 | |
2026.05 | 36.8 | 268 | 258.2 | 410 | 395 | 2,353 | 30.9 | |
2026.05 | 29.5 | 66.3 | 80.4 | 170 | 206 | 1,405 | 36.7 | |
2026.07 | 25.2 | - | - | - | - | - | - | |
2026.05 | 24.7 | 133.1 | 138.4 | 300 | 312 | 1,597 | 25.9 |