Share your thoughts, 1 month free Claude Pro on usSee more

Multi-task Language Understanding on MMLU Pro (Accuracy)

66.3Accuracy

Llama 3.1 Instruct

Updated 4d ago

Evaluation Results

Method	Links
Llama 3.1 Instruct 2025.04		66.3
Llama 3 Instruct 2025.04		63.2
ParamΔ 2025.04		62.1
SPELL 2025.09		60.22
SPELL 2025.09		58.86
Llama 3 Base 2025.04		54
Llama 3.1 Base 2025.04		51.3
SPELL 2025.09		49.78
Qwen2.5-32B 2025.09		48.89
Llama 3.1 Instruct 2025.04		48.6
Qwen2.5-14B 2025.09		46.67
Llama 3 Instruct 2025.04		45.5
ParamΔ 2025.04		45.5
Qwen2.5-7B 2025.09		40.24
Llama 3.1 Base 2025.04		36.4
Llama 3 Base 2025.04		33
Dense 2025.06		19.6
MoB 2025.06		19.1
MiCRo 2025.06		19
Dense 2025.06		11.2
MoB 2025.06		11
MiCRo 2025.06		10.7
MiCRo 2025.06		10.1
Dense 2025.06		9.9
MoB 2025.06		9.8
MiCRo 2025.06		7.9
Dense 2025.06		7.8
MoB 2025.06		7.4