| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| CodeXGLUE | GALLa | Repair Rate15.7 | 21 | 4d ago | |
| CodeFix | GPT-4o | Fix Success Rate85.5 | 20 | 4d ago | |
| HumanEvalFix (test) | WaveCoder-Pro-6.7B | Success Rate (Python)59.1 | 19 | 4d ago | |
| Tufano Medium Abstract 2019 (test) | CoTexT | Top-1 Exact Match Accuracy15.36 | 6 | 4d ago | |
| Tufano Small Abstract 2019 (test) | NSEdit | Top-1 Accuracy24.04 | 6 | 4d ago | |
| DeepFix | PaLM-Coder | Pass@182.1 | 5 | 3d ago | |
| Tufano Medium Concrete 2019 (test) | NSEdit | Top-1 Acc13.46 | 2 | 4d ago | |
| Tufano Small Concrete 2019 (test) | NSEdit | Top-1 Acc23.86 | 2 | 4d ago | |
| DeepFix (test) | - | Pass@1- | 0 | 4d ago |
