Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Evaluation

Benchmarks

Dataset Name	SOTA Method	Metric
Downstream Tasks Zero-shot		Accuracy76	278	3mo ago
Downstream Tasks MMLU, PIQA, Arc-E, Arc-C, Wino, OpenQA		MMLU77.62	218	3mo ago
Eight datasets average	Surrogate-Assisted Layer Contribution Estimation	Accuracy63.54	112	8d ago
Zero-shot Tasks Average		Accuracy68.05	95	1mo ago
PIQA, WinoGrande, HellaSwag, ARC (Easy and Challenge), LAMBADA (test)		Average Accuracy77.07	90	3mo ago
9 diverse tasks zero-shot		Average Accuracy (Zero-shot)73.81	85	7d ago
Zero-shot Evaluation Suite (ARC, Hellaswag, LAMBADA, PIQA, Winogrande)		ARC-C73.39	85	14d ago
Evaluation Tasks Zero-shot Aggregate		Avg. Accuracy79.95	74	14d ago
6 zero-shot downstream tasks		Average Accuracy80.05	70	14d ago
ArcC, ArcE, HS, PiQA, WG (test val)		Average Accuracy76	61	3mo ago
ARC-Easy, ARC-Challenge, OpenBookQA, WinoGrande, PIQA, HellaSwag, MathQA, RTE, BoolQ zero-shot	Original (Dense)	Mean Accuracy71.08	59	2mo ago
ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, RTE, WinoGrande Zero-shot		Accuracy72.9	57	1mo ago
7 tasks zero-shot	BTC-LLM	Mean Accuracy (Zero-shot)72.79	55	12d ago
Zero-shot Evaluation Suite (ARC-c, ARC-e, HellaSwag, PIQA, WinoGrande)		ARC-c Accuracy50.3	52	2d ago
Various Tasks Average (test)		Average Accuracy79.95	38	3mo ago
Evaluation Benchmarks Zero-shot		Average Accuracy71.76	34	2d ago
Downstream Tasks PiQA ARC Hellaswag Winogrande BoolQ		PiQA Accuracy (Zero-shot)84.4	30	1mo ago
Eight tasks zero-shot		Accuracy (Zero-shot)60.31	29	1mo ago
Zero-shot Tasks		Task Avg Score73.99	26	15d ago
ARC-Challenge, ARC-Easy, BoolQ, CrowS-Pairs, OpenBookQA, PIQA, RACE, SiQA, TruthfulQA, WinoGrande zero-shot	HFPrune	ARC-C Accuracy52.1	26	2mo ago
Tasks Zero-shot (mean)		mAcc76.57	25	3mo ago
MobileLLM Evaluation Suite zero-shot	EDGERAZOR	ARC-e69.19	23	27d ago
OpenCLIP 38 datasets		Average Performance67.3	22	3mo ago
StableEval 27 evals	ACED-F2	Average Performance70.9	21	3mo ago
Reasoning Benchmarks Zero-shot (BoolQ, PIQA, HellaSwag, WinoGrande, ARC)		BoolQ Accuracy (Zero-shot)71.1	20	2mo ago

Showing 25 of 53 rows