Share your thoughts, 1 month free Claude Pro on usSee more

Language Model Evaluation on Tulu-3 (dev)

60.85Knowledge Score

Equal

Updated 24d ago

Evaluation Results

Method
Equal 2026.07	60.85	64.55	59.03	53.61	68.58	53.49	60.02
ODM 2026.07	60.25	65.69	59.42	44.22	63.59	54.83	58
Qwen2.5-7B-Instruct 2026.07	60.24	52.36	54.7	68.67	44.36	60.47	56.8
RegMix 2026.07	59.6	68.12	63.37	55.76	58.04	55.94	60.14
CAUSALMIX-S 2026.07	59.35	62.88	58.63	64.43	67.47	60.95	62.28
DMO 2026.07	59.15	63.7	60.62	54.05	70.24	54.35	60.35
Grid 2026.07	59.08	68.77	65.99	61.55	44.92	56.43	59.46
DoReMi 2026.07	58.02	63.2	57.35	57.28	68.21	52.75	59.47
CAUSALMIX-A 2026.07	57.14	64.03	58.51	65.52	68.21	57.65	61.84
Llama-3.1-Tulu-3-8B-SFT 2026.07	51.63	67.37	56.14	57.91	68.39	46.39	57.97
Qwen2.5-0.5B-Instruct 2026.07	29.9	32.14	35.84	34.1	30.31	11.14	28.9
DoReMi 2026.07	29.81	31.7	22.9	33.4	35.3	25.34	29.74
CAUSALMIX-S 2026.07	29.45	31.3	26.37	32.4	38.82	25.95	30.71
Grid 2026.07	29.34	31.84	26.68	35.02	16.64	17.26	26.13
DMO 2026.07	28.86	30.91	22.51	30.68	35.86	26.19	29.17
CAUSALMIX-A 2026.07	28.73	30.63	24.67	30.48	42.51	43.45	33.41
ODM 2026.07	28.65	30.74	24.35	34.62	30.5	12.61	26.91
ODM 2026.07	28.59	30.5	25.18	36.06	33.64	10.28	27.37
DoReMi 2026.07	28.58	31.12	22.14	33.61	39.74	26.56	30.29
RegMix 2026.07	28.57	30.19	27.02	34.32	24.21	16.65	26.82
Grid 2026.07	28.46	31.98	30.41	33.5	20.15	23.38	27.98
Equal 2026.07	28.39	32.98	23.96	32.1	37.89	27.78	30.51
CAUSALMIX-A 2026.07	28.37	31.31	24.67	29.57	38.63	26.93	29.91
CAUSALMIX-S 2026.07	28.31	30.96	27.64	30.97	42.51	36.47	32.81
Equal 2026.07	28.07	29.28	21.59	36.06	46.95	25.09	31.78
DMO 2026.07	28.07	31.88	22.42	26.9	41.59	41.37	32.04
CAUSALMIX-A 2026.07	27.93	29.68	23.56	27.76	43.81	50.92	33.94
DMO 2026.07	27.74	31.75	24.25	31.8	38.63	41.62	32.63
DoReMi 2026.07	27.7	29.66	20.58	34.53	42.88	32.93	31.38
Equal 2026.07	27.65	29.8	23.32	33	33.83	14.57	27.03
ODM 2026.07	27.5	30.53	25.41	33.1	26.06	23.62	27.7
RegMix 2026.07	27.29	30.04	25.53	31.28	17.01	32.56	27.28
CAUSALMIX-S 2026.07	27.27	30.23	27.77	34.94	34.94	14.81	27.85
RegMix 2026.07	24.9	29.94	30.25	25.46	25.32	22.52	26.4
Grid 2026.07	23.94	24.4	29.67	23.81	17.93	22.64	31.17