| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| CodeXGLUE | GALLa | Repair Rate15.7 | 21 | 1mo ago | |
| CodeFix | GPT-4o | Fix Success Rate85.5 | 20 | 1mo ago | |
| HumanEvalFix (test) | WaveCoder-Pro-6.7B | Success Rate (Python)59.1 | 19 | 1mo ago | |
| Tufano Medium Abstract 2019 (test) | CoTexT | Top-1 Exact Match Accuracy15.36 | 6 | 1mo ago | |
| Tufano Small Abstract 2019 (test) | NSEdit | Top-1 Accuracy24.04 | 6 | 1mo ago | |
| SWE-bench Lite | ADARUBRIC-DA | r0.77 | 5 | 25d ago | |
| DeepFix | PaLM-Coder | Pass@182.1 | 5 | 1mo ago | |
| Tufano Medium Concrete 2019 (test) | NSEdit | Top-1 Acc13.46 | 2 | 1mo ago | |
| Tufano Small Concrete 2019 (test) | NSEdit | Top-1 Acc23.86 | 2 | 1mo ago | |
| DeepFix (test) | - | Pass@1- | 0 | 1mo ago |
