Share your thoughts, 1 month free Claude Pro on usSee more

Multi-task Language Understanding on MMLU (test)

90.46Normalized Accuracy

Oracle

Updated 1mo ago

Evaluation Results

Method
Oracle 2024.08	90.46	1.75	-	-	-	-
Full Reasoning 2026.01	81.6	105.67	1,465	-	-	-
SyncThink 2026.01	77.99	61.59	852	-	-	-
MASPRM-7B 2025.10	75.2	-	-	0.2	12.4	-
MASPRM-7B 2025.10	74.8	-	-	0.7	35	-
SFT-DPO + Merged 2026.02	72.93	-	-	-	-	-
MASPRM-7B 2025.10	72.4	-	-	0.2	15	-
SFT + OGPSA 2026.02	70.93	-	-	-	-	-
Gemma2-9B (Baseline) 2026.02	70.5	-	-	-	-	-
SFT-DPO + Merged 2026.02	70.21	-	-	-	-	-
MASPRM-1.5B 2025.10	68.8	-	-	0.2	12.4	-
FP16 2026.01	67.31	-	-	-	-	-
MASPRM-7B 2025.10	67.2	-	-	0.3	15	-
ScaleBITS + RTN 2026.02	67.12	-	-	-	-	-
FP16 2025.12	66.5	-	-	-	-	-
ORM-1.5B 2025.10	66.2	-	-	0.2	12.1	-
SELECTLLM 2024.08	65.81	4.78	-	-	-	-
Top-s LLMs 2024.08	65.75	16.4	-	-	-	-
SELECTLLM 2024.08	65.68	4.78	-	-	-	-
SELECTLLM 2024.08	65.68	4.78	-	-	-	-
RTN-g128 2026.02	65.52	-	-	-	-	-
Llama 3-8B 2024.12	65.2	-	-	-	-	-
Llama 3-8B E8T2 2024.12	64.1	-	-	-	-	-
Llama 3-8B E8T2 2024.12	64	-	-	-	-	-
HeRo-Q 2026.01	63.8	-	-	-	-	-
SELECTLLM 2024.08	63.52	2.97	-	-	-	-
FP16 2025.12	63.4	-	-	-	-	-
Full Reasoning 2026.01	62.8	99.38	2,076	-	-	-
Unilogit+KL 2025.05	62.6	-	-	-	-	-
Llama 3-8B 2024.12	62.1	-	-	-	-	-
Full Reasoning 2026.01	61.15	73	1,676	-	-	-
All LLMs 2024.08	60.92	16.4	-	-	-	-
Baseline 2025.05	60.8	-	-	-	-	-
ME+GD 2025.05	60.8	-	-	-	-	-
SpinQuant 2026.01	60.5	-	-	-	-	-
Qwen2.5-1.5B-Instruct 2025.10	60.5	-	-	0.2	12.4	-
MASPRM-1.5B 2025.10	60.4	-	-	0.8	35	-
LLM-Blender 2024.08	60.27	16.4	-	-	-	-
CALM 2025.12	60.2	-	-	-	-	-
SyncThink 2026.01	59.98	41.91	872	-	-	-
OmniQuant 2026.01	59.8	-	-	-	-	-
NPO+KL 2025.05	59.6	-	-	-	-	-
Undial+KL 2025.05	59.6	-	-	-	-	-
Unilogit+KL 2025.05	59.6	-	-	-	-	-
Qwen2.5-1.5B-Instruct 2025.10	59.4	-	-	0.6	35	-
Undial+KL 2025.05	59.1	-	-	-	-	-
SyncThink 2026.01	59.1	28.13	643	-	-	-
MASPRM-1.5B 2025.10	58.7	-	-	0.3	15	-
SpinQuant 2025.12	58.5	-	-	-	-	-
ORM-1.5B 2025.10	58.5	-	-	0.3	15	-
MASPRM-1.5B 2025.10	58.3	-	-	0.3	15	-
Random 2024.08	58.2	8.27	-	-	-	-
ScaleBITS + RTN 2026.02	57.89	-	-	-	-	-
CALM 2025.12	57.5	-	-	-	-	-
GA+KL 2025.05	57.3	-	-	-	-	-
SmoothQuant 2025.12	57.2	-	-	-	-	-
SpinQuant 2025.12	55	-	-	-	-	-
Qwen2.5-1.5B-Instruct 2025.10	55	-	-	0.3	15	-
Qwen2.5-1.5B-Instruct 2025.10	54.5	-	-	0.3	15	-
AWQ 2025.12	54.2	-	-	-	-	-
SmoothQuant 2025.12	53.8	-	-	-	-	-
No Reasoning 2026.01	53.26	12.01	276	-	-	-
Qwen2.5-1.5B-Instruct 2025.10	52.7	-	-	0.3	15	-
AWQ 2025.12	50.5	-	-	-	-	-
Qwen2.5-1.5B-Instruct 2025.10	48.4	-	-	0.1	3	-
FP16 2026.01	46.38	-	-	-	-	-
GPTQ 2025.12	45	-	-	-	-	-
GPTQ 2025.12	45	-	-	-	-	-
HeRo-Q 2026.01	42.5	-	-	-	-	-
AWQ 2026.01	40.23	-	-	-	-	-
AWQ 2026.01	39.5	-	-	-	-	-
SpinQuant 2026.01	39.1	-	-	-	-	-
OmniQuant 2026.01	36.5	-	-	-	-	-
GPTQ 2026.01	25.12	-	-	-	-	-
RTN-g128 2026.02	23.96	-	-	-	-	-
GPTQ 2026.01	14.2	-	-	-	-	-
Soft Prompt 2026.02	-	-	-	-	-	26.5
Soft Prompt + L2-Reg 2026.02	-	-	-	-	-	27
Soft Prompt + PDLoss 2026.02	-	-	-	-	-	28.8
ACT 2026.02	-	-	-	-	-	28.2
Prefix-Tuning 2026.02	-	-	-	-	-	29.5
DPC 2026.02	-	-	-	-	-	31.8
TSLoss 2026.02	-	-	-	-	-	32.4
Soft Prompt 2026.02	-	-	-	-	-	40.2
Soft Prompt + L2-Reg 2026.02	-	-	-	-	-	40.8
Soft Prompt + PDLoss 2026.02	-	-	-	-	-	42.5
ACT 2026.02	-	-	-	-	-	41.5
Prefix-Tuning 2026.02	-	-	-	-	-	43.1
DPC 2026.02	-	-	-	-	-	45.2
TSLoss 2026.02	-	-	-	-	-	44.8
Soft Prompt 2026.02	-	-	-	-	-	52.5
Soft Prompt + L2-Reg 2026.02	-	-	-	-	-	52.8
Soft Prompt + PDLoss 2026.02	-	-	-	-	-	54.2
ACT 2026.02	-	-	-	-	-	53.5
Prefix-Tuning 2026.02	-	-	-	-	-	55
DPC 2026.02	-	-	-	-	-	56.5
TSLoss 2026.02	-	-	-	-	-	56.8
Soft Prompt 2026.02	-	-	-	-	-	58
Soft Prompt + L2-Reg 2026.02	-	-	-	-	-	58.5
Soft Prompt + PDLoss 2026.02	-	-	-	-	-	60

Showing 100 of 123 rows