| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| Real-FIM-Eval | SFT | Exact Match (EM)4.4 | 16 | 1mo ago | |
| SAFIM control | ORPO | Exact Match (EM)41 | 16 | 1mo ago | |
| SAFIM block v2 | base | Exact Match (EM)37.6 | 16 | 1mo ago | |
| SAFIM block | base | Exact Match (EM)30.9 | 16 | 1mo ago | |
| SAFIM API | SFT | Exact Match (%)72.6 | 16 | 1mo ago | |
| HE-Inf random-span | base | Exact Match (EM)50.4 | 16 | 1mo ago | |
| HE-Inf multi-line | base | Exact Match (EM)27.1 | 16 | 1mo ago | |
| HE-Inf single-line | SFT | Exact Match (EM)57.3 | 16 | 1mo ago | |
| Delulu | ORPO | EM (%)62.4 | 16 | 1mo ago | |
| FIM-Tasks Single-Line Infilling (test) | CodeLlama-Base | Python Score68.3 | 7 | 4mo ago |
