Zero-shot Evaluation

Benchmarks

Dataset Name	SOTA Method	Metric
Downstream Tasks Zero-shot		Accuracy76	278	4d ago
Downstream Tasks MMLU, PIQA, Arc-E, Arc-C, Wino, OpenQA		MMLU77.62	218	4d ago
PIQA, WinoGrande, HellaSwag, ARC (Easy and Challenge), LAMBADA (test)		Average Accuracy77.07	90	4d ago
Eight datasets average	Surrogate-Assisted Layer Contribution Estimation	Accuracy63.54	87	4d ago
ArcC, ArcE, HS, PiQA, WG (test val)		Average Accuracy76	61	4d ago
Various Tasks Average (test)		Average Accuracy79.95	38	4d ago
Evaluation Tasks Zero-shot Aggregate		Avg. Accuracy75.41	33	4d ago
Tasks Zero-shot (mean)		mAcc76.57	25	4d ago
OpenCLIP 38 datasets		Average Performance67.3	22	4d ago
StableEval 27 evals	ACED-F2	Average Performance70.9	21	4d ago
6 zero-shot downstream tasks		Average Accuracy80.05	19	4d ago
DCLM CORE V2	Union (Random)	CORE_V2 Score48	17	4d ago
Zero-shot Evaluation Suite		ARCC60.5	14	4d ago
lm-evaluation-harness (SciQ, ARC-E, ARC-C, LogiQA, OBQA, BoolQ, HellaSwag, PIQA, WinoGrande) zero-shot	DsDm	SciQ Accuracy68.2	13	4d ago
DataComp full (test)	TuneCLIP	Mean Zero-shot Accuracy63.47	8	4d ago
Reasoning tasks		Reasoning Accuracy70.7	7	4d ago
Downstream NLU tasks 1.0 (eval)	Random Sampling KD (Ours 12+)	0-shot Score57.9	6	4d ago
Zero-shot Downstream Tasks (Arc-e, PIQA, Hellaswag, OpenBookQA, Winogrande, MMLU, BoolQ) Llama-1B Benchmark Suite (test)	Peri-LN	Arc-e Accuracy31.63	5	4d ago
Non-reasoning tasks		Accuracy (Zero-shot Non-reasoning)70.8	4	4d ago
GPT-3 Evaluation Suite (LAMBADA, TriviaQA, WebQs, PIQA, RACE-h, BoolQ) 1.3B various (test val)	GPT-3 1.3B (Original)	Overall Accuracy44.4	3	4d ago

Showing 20 of 20 rows