Share your thoughts, 1 month free Claude Pro on usSee more

Multi-task Language Modeling on MergeBench

39.56Instruction Score

Task-specific FT

Updated 1mo ago

Evaluation Results

Method	Links
Task-specific FT 2026.02		39.56	69.83	44.33	41.73	80.46	55.18
TIES + OrthoMerge-C 2026.02		25.32	55.8	40.21	42.26	44.24	41.57
TIES 2026.02		20.89	50.8	39.61	42.11	42.38	39.16
TIES + OrthoMerge-G 2026.02		20.33	51.48	40.28	42.08	42.51	39.34
TSV-M + OrthoMerge-C 2026.02		19.59	55.88	41.69	42.32	50.87	42.07
TSV-M 2026.02		19.4	55.72	40.44	42.2	49.82	41.52
TSV-M + OrthoMerge-G 2026.02		18.67	51.71	40.65	41.89	47.23	40.03
TA + OrthoMerge-G 2026.02		11.09	43.9	37.96	42.06	42.41	35.48
TA 2026.02		10.53	40.4	37.22	42.26	40.4	34.16
TA + OrthoMerge-C 2026.02		9.8	40.03	37.75	42.29	41.69	34.31
Llama-3.2-3B 2026.02		7.58	28.51	27.44	40.72	31.41	27.18