Share your thoughts, 1 month free Claude Pro on usSee more

Multi-task Language Understanding on MMLU Pro (Accuracy)

96.8Accuracy

Pass@100

Updated 22d ago

Evaluation Results

Method	Links
Pass@100 2026.04		96.8
Pass@100 2026.04		92
Logistic 2026.04		91.8
FUSE 2026.04		91.4
Naive Bayes 2026.04		91.4
Weaver 2026.04		90.2
Naive Ensemble 2026.04		87
OBV (Oracle Best Verifier) 2026.04		85.6
Qwen3-30B-A3B 2026.05		81.11
NanoV3-30B* 2026.05		78.86
NanoV3 Elastic-30B 2026.05		78.63
NanoV3 Elastic-23B 2026.05		76.07
Majority Vote 2026.04		74.4
Pass@1 2026.04		69.9
FUSE 2026.04		69.8
NanoV3 Elastic-12B 2026.05		68.28
Naive Ensemble 2026.04		67.2
Weaver 2026.04		67.2
OBV (Oracle Best Verifier) 2026.04		66.5
Llama 3.1 Instruct 2025.04		66.3
Naive Bayes 2026.04		66
Logistic 2026.04		65.2
Llama 3 Instruct 2025.04		63.2
ParamΔ 2025.04		62.1
SPELL 2025.09		60.22
SPELL 2025.09		58.86
Majority Vote 2026.04		56.4
Llama 3 Base 2025.04		54
Llama 3.1 Base 2025.04		51.3
SPELL 2025.09		49.78
Qwen2.5-32B 2025.09		48.89
Llama 3.1 Instruct 2025.04		48.6
Qwen2.5-14B 2025.09		46.67
Pass@1 2026.04		46.6
Llama 3 Instruct 2025.04		45.5
ParamΔ 2025.04		45.5
Qwen2.5-7B 2025.09		40.24
Llama 3.1 Base 2025.04		36.4
Llama 3 Base 2025.04		33
Dense 2025.06		19.6
Reg. MoE 2026.05		19.3
MoB 2025.06		19.1
MiCRo 2025.06		19
OLMoE 2026.05		18.7
EMO 2026.05		18.5
Reg. MoE 2026.05		15.8
EMO 2026.05		15.5
Dense 2026.05		12.2
Dense 2025.06		11.2
MoB 2025.06		11
MiCRo 2025.06		10.7
MiCRo 2025.06		10.1
Dense 2025.06		9.9
MoB 2025.06		9.8
MiCRo 2025.06		7.9
Dense 2025.06		7.8
MoB 2025.06		7.4