Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

MMLU-Pro

Benchmarks

Task NameDataset NameSOTA ResultTrend
Class-level Continual LearningMMLU-PRO
Average Accuracy (AA)50.72
56
Multiple Choice Question AnsweringMMLU-PRO zero-shot
Accuracy84.29
51
Capability Self-AssessmentMMLU-Pro Science
M-F169.1
40
Multiple-choice Question AnsweringMMLU-Pro Chem.
Accuracy72.8
40
KnowledgeMMLU-Pro 5-shot
Knowledge Score (5-shot)44.65
37
HealthMMLU-Pro Health (FR) X (test)
Accuracy66.08
35
Mathematical ReasoningMMLU-Pro Math
Accuracy89.79
26
Multi-task Language UnderstandingMMLU-Pro
Best Accuracy71.4
25
Hallucination evaluationMMLU-Pro Law (test)
HALL%12.1
21
ScienceMMLU-Pro (test)
Accuracy41.9
18
Academic ReasoningMMLU-Pro
Pass@150.7
15
Medical Question AnsweringMMLU-Pro Health
Accuracy60.76
12
Hardened Language UnderstandingMMLU-Pro (test)
Accuracy (MMLU-Pro Test)23.4
11
Medical ReasoningMMLU-Pro Biology English
Accuracy77.7
11
Language UnderstandingMMLU-Pro (test)
MMLU-Pro (test) Accuracy23.6
11
Multi-task Language UnderstandingMMLU-Pro AceReason (Reduced)
Accuracy71.1
10
Multi-task Language UnderstandingMMLU-Pro AceReason (Complete)
Accuracy (MMLU-Pro AceReason)76.5
10
Language UnderstandingMMLU-Pro 80 (test)
Pass@138.68
10
General Knowledge ReasoningMMLU-Pro (test)
Accuracy37.72
10
General Knowledge ReasoningMMLU-Pro
BCA91.6
9
Math-reasoningMMLU-Pro
pass@140.3
8
Ranking Consistency AnalysisMMLU-Pro Nutrition health
Spearman Correlation0.78
8
Ranking Consistency AnalysisMMLU-Pro Medical genetics health
Spearman Correlation0.42
8
Ranking Consistency AnalysisMMLU-Pro health Human aging
Spearman Correlation0.62
8
Ranking Consistency AnalysisMMLU-Pro health Virology
Spearman Correlation0.55
8
Showing 25 of 57 rows