Share your thoughts, 1 month free Claude Pro on usSee more

Fill-in-the-Middle Code Completion on Real-FIM-Eval

4.4Exact Match (EM)

SFT

Updated 1mo ago

Evaluation Results

Method	Links
SFT 2026.06		4.4	33.9
SFT 2026.06		4.2	33.1
ORPO 2026.06		3.8	34.3
SFT 2026.06		3.7	32.5
ORPO 2026.06		3.7	33.8
SFT 2026.06		3.6	31.8
ORPO 2026.06		3.3	32.9
ORPO 2026.06		3.3	33.3
DPO 2026.06		2.4	34
DPO 2026.06		2.3	33.6
DPO 2026.06		1.7	19.3
DPO 2026.06		1.2	18.9
base 2026.06		0.4	18.5
base 2026.06		0.4	18.2
base 2026.06		0.2	14.3
base 2026.06		0.2	14