Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Evaluation

Benchmarks

Dataset Name	SOTA Method	Metric
Downstream Tasks Zero-shot		Accuracy76	278	1mo ago
Downstream Tasks MMLU, PIQA, Arc-E, Arc-C, Wino, OpenQA		MMLU77.62	218	1mo ago
Zero-shot Tasks Average		Accuracy68.05	95	4d ago
PIQA, WinoGrande, HellaSwag, ARC (Easy and Challenge), LAMBADA (test)		Average Accuracy77.07	90	1mo ago
Eight datasets average	Surrogate-Assisted Layer Contribution Estimation	Accuracy63.54	87	1mo ago
ArcC, ArcE, HS, PiQA, WG (test val)		Average Accuracy76	61	1mo ago
ARC-Easy, ARC-Challenge, OpenBookQA, WinoGrande, PIQA, HellaSwag, MathQA, RTE, BoolQ zero-shot	Original (Dense)	Mean Accuracy71.08	59	1mo ago
ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, RTE, WinoGrande Zero-shot		Accuracy72.9	57	11d ago
Evaluation Tasks Zero-shot Aggregate		Avg. Accuracy75.41	39	22d ago
Various Tasks Average (test)		Average Accuracy79.95	38	1mo ago
6 zero-shot downstream tasks		Average Accuracy80.05	31	8d ago
Downstream Tasks PiQA ARC Hellaswag Winogrande BoolQ		PiQA Accuracy (Zero-shot)84.4	30	8d ago
ARC-Challenge, ARC-Easy, BoolQ, CrowS-Pairs, OpenBookQA, PIQA, RACE, SiQA, TruthfulQA, WinoGrande zero-shot	HFPrune	ARC-C Accuracy52.1	26	1mo ago
Tasks Zero-shot (mean)		mAcc76.57	25	1mo ago
OpenCLIP 38 datasets		Average Performance67.3	22	1mo ago
StableEval 27 evals	ACED-F2	Average Performance70.9	21	1mo ago
7 tasks zero-shot	BTC-LLM	Mean Accuracy (Zero-shot)72.79	20	8d ago
Reasoning Benchmarks Zero-shot (BoolQ, PIQA, HellaSwag, WinoGrande, ARC)		BoolQ Accuracy (Zero-shot)71.1	20	1mo ago
DCLM CORE V2	Union (Random)	CORE_V2 Score48	17	1mo ago
Downstream tasks average		Avg Zero-shot Accuracy81	16	8d ago
Zero-shot Evaluation Suite		ARCC60.5	14	1mo ago
lm-evaluation-harness (SciQ, ARC-E, ARC-C, LogiQA, OBQA, BoolQ, HellaSwag, PIQA, WinoGrande) zero-shot	DsDm	SciQ Accuracy68.2	13	1mo ago
Zero-shot Tasks		Task Avg Score73.99	10	8d ago
Winogrande, OBQA, Hellaswag, Boolq, ARC, RTE (test)	BTC-LLM	Winogrande Accuracy76.07	9	8d ago
Evaluation Suite (PiQA, LAMBADA, ARC, HellaSwag) zero-shot		PiQA Accuracy62.47	9	24d ago

Showing 25 of 37 rows