Autonomous Software Engineering on SWE-Bench Pro (test)
32.6Resolution Rate
Sonnet
Updated 2mo ago
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
2026.05 | 32.6 | 756 | 19 | 774 | 0.6 | 6 | 100 | |
2026.05 | 32.4 | 727 | 17 | 727 | 1.1 | 17 | 98.5 | |
2026.05 | 32 | 729 | 19 | 748 | 0.5 | 4 | 100 | |
2026.05 | 31.5 | 730 | 18 | 730 | 1 | 14 | 97.9 | |
2026.05 | 30.4 | 840 | 6 | 840 | 1.9 | 27 | 99 | |
2026.05 | 30 | 836 | - | 836 | 3.8 | - | - |