| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| SWE-Bench C# | Resolve Rate47.3 | 18 | 2mo ago | ||
| SWE-bench Verified | DELM | Average Success Rate @178 | 10 | 1mo ago | |
| SWE-Bench Verified | Mini SWE Agent + LaMR | Rounds Taken25.8 | 6 | 2mo ago | |
| SWE-Bench hard problem subset | LeanEvolve | Add. Solved10.67 | 5 | 1mo ago | |
| SWE-bench Lite (test) | Resolution Rate57.3333 | 2 | 2mo ago |
