SAFIM
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Fill-in-the-Middle Code Completion | SAFIM control | Exact Match (EM)41 | 16 | |
| Fill-in-the-Middle Code Completion | SAFIM block v2 | Exact Match (EM)37.6 | 16 | |
| Fill-in-the-Middle Code Completion | SAFIM block | Exact Match (EM)30.9 | 16 | |
| Fill-in-the-Middle Code Completion | SAFIM API | Exact Match (%)72.6 | 16 | |
| Block Infilling | SAFIM | Pass@169.47 | 6 | |
| Code Completion | SAFIM | Pass@154.1 | 3 | |
| Reasoning failure prediction and recovery | SAFIM L3 | Accuracy81 | 2 | |
| Reasoning failure prediction and recovery | SAFIM L1 | Accuracy79 | 2 |
Showing 8 of 8 rows