Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Reasoning

Benchmarks

Dataset Name	SOTA Method	Metric
Reasoning Suite Zero-shot (PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c) (val test)		Average Accuracy76.55	305	1mo ago
Reasoning Suite (ARC-e, ARC-c, HellaSwag, PIQA, Winogrande) zero-shot	RIA+SQ+VC+EBFT	Average Reasoning Score6,540	125	1mo ago
PIQA		PIQA Zero-shot Accuracy82.54	115	1mo ago
WinoGrande		Accuracy80.51	107	1mo ago
Evaluation Suite Zero-shot (OpenbookQA, ARC-e, ARC-c, WinoGrande, HellaSwag, PIQA, MathQA)	Meta-Llama-3-8B Dense	Average Accuracy69.99	97	26d ago
ARC-Easy zero-shot	WeDLM-8B	Zero-shot Accuracy97.43	94	1mo ago
9 Common Sense Reasoning Tasks (WinoGrande, SocialIQA, LAMBADA, MMLU, ARC-Easy, ARC-Challenge, HellaSwag, OpenBookQA, PIQA) Average		Accuracy72.7	65	2mo ago
Reasoning Tasks (BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA) Zero-shot		BoolQ Accuracy (Zero-shot)82.813	64	1mo ago
ZeroShot 7		Accuracy71.57	56	2mo ago
Zero-Shot Reasoning Tasks (ARC-C, ARC-E, BoolQ, Hella, OBQA, PIQA, SIQA, Wino)		ARC-C Accuracy65.53	54	3mo ago
ARC Challenge		Zero-Shot Accuracy60.41	53	1mo ago
HellaSwag		Accuracy76.3	53	1mo ago
ARC-e, PIQA, OpenbookQA, Winogrande, HellaSwag, MathQA		Average Accuracy57	40	1mo ago
ARC-e, Winogrande, HellaSwag, PIQA		Normalized Avg Accuracy77.2	36	4mo ago
Reasoning Benchmarks Average		Average Accuracy75.33	34	1mo ago
MathQA		Accuracy28.4	26	3mo ago
OpenbookQA		Accuracy44	26	3mo ago
Downstream Tasks (LMB, PIQA, HellaSwag, OPQA, ARC)	FLT	LAMBADA (LMB) Accuracy32.75	22	2mo ago
benchmark datasets (PIQA, HeSw, ARC-e, ARC-c, OBQA, Race, WSC273, LAMBADA, MMLU) Zero-shot	LLaMA-2-7B-hf	PIQA78.07	21	4mo ago
Zero-shot Reasoning Tasks (ARC-e, ARC-c, BoolQ, Hellaswag, WinoGrande, MMLU)		ARC-e Accuracy (Zero-shot)77.1	20	1mo ago
Reasoning Suite PiQA, LAMBDA, ARC, HellaSwag	NSA	PiQA Score62.69	20	4mo ago
Common Sense Reasoning Suite (OpenbookQA, ARC-E, ARC-C, Winogrande, HellaSwag, PIQA, MathQA)		OpenbookQA Accuracy28	18	18d ago
Reasoning Tasks (PIQA, ARC-e, HS, WG) Zero-shot	Llama2-13B (W16A16)	PIQA Accuracy80.41	18	4mo ago
HellaSwag zero-shot 200 items	HQMQ s192 r6 + Med3x	Accuracy68	17	1mo ago
Reasoning Tasks ARC-C, BoolQ, HellaSwag, MathQA, MMLU, Race		ARC-C Accuracy50.43	14	1mo ago

Showing 25 of 37 rows