Zero-shot Question Answering

Benchmarks

Dataset Name	SOTA Method	Metric
QA Suite (PIQA, Winogrande, HellaSwag, ARC-E, ARC-C, LAMBADA) Zero-shot	QuaRot	PIQA Accuracy78.24	56	2mo ago
Evaluation Suite (WinoGrande, PiQA, HellaSwag, ARC-E, ARC-C) Zero-shot	PocketLLM	WinoGrande Accuracy69.39	35	1mo ago
AveQA	AMP	Accuracy57.7	25	4mo ago
BoolQ, Winogrande, PIQA, OpenBookQA, HellaSwag, ARC-Easy, ARC-Challenge Zero-shot		BoolQ Accuracy (Zero-shot)83.76	21	4mo ago
Benchmarks (ArcC, ArcE, PiQA, Wino) Zero-shot		ARC-C Accuracy43.43	17	4mo ago
Downstream Reasoning Tasks ARC-c, ARC-e, BoolQ, HellaSwag, MMLU, OpenBookQA, PIQA, Winogrande		ARC-c Accuracy (Zero-shot)58.4	15	4mo ago
OpenBookQA, PIQA, RACE, SciQ, WinoGrande	Hyperloop	Accuracy54.6	12	3mo ago
Reasoning Suite (OBQA, PIQA, ARC-e, ARC-c, Hella, Wino) Zero-shot		Accuracy on OBQA (Zero-shot)46.8	8	1mo ago
Commonsense Reasoning Suite (PIQA, WinoGrande, HellaSwag, ARC) Zero-shot Llama-2-70B		PIQA Accuracy (Zero-shot)82.7	7	3mo ago
OpenBookQA	Yat (sb+α)	Accuracy28.4	5	2mo ago
ARC Easy	GELU	Accuracy33.53	5	2mo ago
Reasoning Benchmarks ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, WinoGrande		ARC-C Accuracy60.15	3	1mo ago
MMLU zero-shot	Pre-trained	Average Loss1.03	2	4mo ago

Showing 13 of 13 rows