Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Evaluation

Benchmarks

Dataset Name	SOTA Method	Metric
Downstream Tasks Zero-shot		Accuracy76	278	4mo ago
7 tasks zero-shot	BTC-LLM	Mean Accuracy (Zero-shot)72.79	223	1mo ago
Downstream Tasks MMLU, PIQA, Arc-E, Arc-C, Wino, OpenQA		MMLU77.62	218	4mo ago
Arc-Challenge, Arc-Easy, HellaSwag, LAMBADA-openai, LAMBADA-standard, PIQA, and WinoGrande Zero-Shot		Accuracy (0-shot)76.71	123	1mo ago
Eight datasets average	Surrogate-Assisted Layer Contribution Estimation	Accuracy63.54	112	2mo ago
PIQA, WinoGrande, HellaSwag, ARC (Easy and Challenge), LAMBADA (test)		Average Accuracy77.07	102	16d ago
Zero-shot Tasks Average		Accuracy68.05	95	16d ago
9 diverse tasks zero-shot		Average Accuracy (Zero-shot)73.81	85	1mo ago
Zero-shot Evaluation Suite (ARC, Hellaswag, LAMBADA, PIQA, Winogrande)		ARC-C73.39	85	2mo ago
Zero-shot Evaluation Suite (ARC-c, ARC-e, HellaSwag, PIQA, WinoGrande)		ARC-c Accuracy59.3	82	1mo ago
Evaluation Tasks Zero-shot Aggregate		Avg. Accuracy79.95	74	2mo ago
6 zero-shot downstream tasks		Average Accuracy80.05	70	2mo ago
ArcC, ArcE, HS, PiQA, WG (test val)		Average Accuracy76	61	4mo ago
Zero-shot Benchmarks Average		Average Accuracy68.6	60	1mo ago
ARC-Easy, ARC-Challenge, OpenBookQA, WinoGrande, PIQA, HellaSwag, MathQA, RTE, BoolQ zero-shot	Original (Dense)	Mean Accuracy71.08	59	4mo ago
ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, RTE, WinoGrande Zero-shot		Accuracy72.9	57	3mo ago
Evaluation Benchmarks Zero-shot		Average Accuracy71.76	55	1mo ago
Various Tasks Average (test)		Average Accuracy79.95	38	4mo ago
Benchmarks 0-shot		0-shot Average Score73.33	33	16d ago
Downstream Tasks PiQA ARC Hellaswag Winogrande BoolQ		PiQA Accuracy (Zero-shot)84.4	30	3mo ago
Eight tasks zero-shot		Accuracy (Zero-shot)60.31	29	2mo ago
Zero-shot Evaluation Suite		Average Score77.28	29	1mo ago
Zero-shot Tasks		Task Avg Score73.99	26	2mo ago
ARC-Challenge, ARC-Easy, BoolQ, CrowS-Pairs, OpenBookQA, PIQA, RACE, SiQA, TruthfulQA, WinoGrande zero-shot	HFPrune	ARC-C Accuracy52.1	26	4mo ago
Tasks Zero-shot (mean)		mAcc76.57	25	4mo ago

Showing 25 of 63 rows