Share your thoughts, 1 month free Claude Pro on usSee more

Language Understanding and Reasoning

Benchmarks

Dataset Name	SOTA Method	Metric
LLM Evaluation Suite MMLU, GSM8k, HellaSwag, WinoGrande		MMLU Score64.43	31	4mo ago
MMLU Pro	AIMMerging	Overall Score67.8	20	2mo ago
LLM Evaluation Suite ARC-e, ARC-c, HellaSwag, OBQA, WinoGrande, MathQA, PIQA		Average Accuracy64.01	19	4mo ago
General Capability Suite (MMLU, TruthfulQA, HellaSwag, ARC-Easy) (test)	Unlearn-Smooth	MMLU Score0.082	16	2mo ago
MMLU, PIQA, HellaSwag, WinoGrande, ARC-Challenge		MMLU (5s)46.45	13	4mo ago
LM-Evaluation-Harness ARC-c, ARC-e, BoolQ, HellaS., MMLU, OBQA, PIQA, WG		ARC-c Accuracy58.4	12	4mo ago
German NLP Benchmark Suite (MMLU, ARC-C, ARC-E, H-Swag, LAMBADA, OBQA)	Qwen3-1.7B-Base	MMLU34.17	10	2mo ago
MMLU, BoolQ, ARC-e, PIQA, Hellaswag, OBQA, Winogrande (test)	Pre-LN + LayerNorm Scaling	MMLU28.69	10	4mo ago
ARC-Challenge, ARC-Easy, BoolQ, HellaSwag (HS), OpenBookQA (OBQA), PIQA, Winogrande (WG), MMLU (test)		ARC-Challenge Accuracy53.16	9	2mo ago
Language Understanding Evaluation Suite (Arc-c, Arc-e, BoolQ, COPA, MMLU, OBQA, PIQA, RTE, Winogrande) Zyda2 calibration (test)	Moonlight	ARC-c58.28	6	4mo ago
ARC-C, ARC-E, HellaSwag, LAMBADA, PIQA, WinoGrande (test)		Mean Score56.81	5	1mo ago
OLMES Standard	OLMo-2-0425-1B	ARC-Easy Accuracy75.9	5	4mo ago
Combined (GSM8k, MATH500, MAWPS, SVAMP, AQuA, GLUE, CSQA, OBQA)	MoSLoRA	Average Score72.94	5	4mo ago
PIQA, HellaSwag, LAMBADA, ARC-Easy, ARC-Challenge, SciQ, RACE, MMLU Zero-shot Benchmarks	ROCKET-ActCost	Average Accuracy (%)53.1	2	26d ago
Downstream Task Suite (PIQA, ARC-e, HellaSwag, GPQA, Lambada, MMLU, BBH)	DAG-MoE-l	PIQA50.67	2	1mo ago
Mixtral-8x7B Zero-shot Suite: Arc C, BoolQ, Lambada, PIQA, Winogrande v0.1-Instruct		Arc C Accuracy65.7	2	3mo ago

Showing 16 of 16 rows