| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| SWE-bench Verified | Percentage Resolved87.6 | 71 | 1mo ago | ||
| SWE-bench 50 tasks sampled | Dense | Resolved Count12 | 20 | 1mo ago | |
| Terminal Bench 2.0 | Pass@159.1 | 18 | 1mo ago | ||
| TerminalBench 2 | Pass Rate81.8 | 17 | 3mo ago | ||
| SWE-Bench Verified | Pass@179.6 | 17 | 1mo ago | ||
| Dialo-SWE-Bench | schema-guided agent | Resolution Rate58.8 | 15 | 1mo ago | |
| Project dev (test) | Tau^20.881 | 13 | 4mo ago | ||
| SWE-Bench Multilingual | MiMo-V2-Flash | Accuracy71.7 | 13 | 1mo ago | |
| SWE-Bench Lite | CONTEXTRL | Solve Rate24 | 9 | 1mo ago | |
| SWE-bench Pro | Pass@152.6 | 9 | 1mo ago | ||
| TerminalBench | LongCat-Flash-Lite | Accuracy0.3375 | 7 | 3mo ago | |
| SWE-Bench | LongCat-Flash-Lite | Accuracy54.4 | 7 | 3mo ago | |
| R&D Coding Benchmark | Pass Rate80 | 6 | 1mo ago | ||
| LiveCodeBench | ALIVE-Self | Pass@156 | 6 | 4mo ago | |
| SWE-bench-Pro | Resolved Rate64.3 | 5 | 1mo ago | ||
| SWE-bench Multilingual | pass@173.3 | 5 | 1mo ago | ||
| U-Artifacts | Pass@157.8 | 5 | 4mo ago | ||
| AInstein-SWE-Bench | Pass@142.8 | 5 | 4mo ago | ||
| Multi-SWE-Bench | Pass@144.3 | 5 | 4mo ago | ||
| MSWE-Bench | SAR | Score31.3 | 4 | 18d ago | |
| SWE-Bench | SAR | Score59 | 4 | 18d ago | |
| SWE-Bench Pro (public) | Solve Rate69.1 | 4 | 1mo ago | ||
| DevEval | Solve Rate94.8 | 4 | 1mo ago | ||
| Coding unseen tasks (test) | SGE | Pass@129.2 | 3 | 4mo ago | |
| TerminalBench 1.0 | SAR | Score0.344 | 2 | 18d ago |
