SOTA Zero-shot Classification benchmarks and papers with code

Benchmarks

Dataset Name	SOTA Method	Metric
16 datasets average	CLIP	Zero-shot Accuracy (%)69.8	90	1mo ago
Downstream Tasks Zero-shot (BoolQ, HellaSwag, WinoGrande, ARC-e, ARC-c, PIQA, OBQA)		BoolQ Accuracy81.65	87	2d ago
CIFAR100	CLIP-PZSL	Top-1 Accuracy74.83	65	16d ago
CIFAR10		Top-1 Clean Acc91.8	62	16d ago
EleutherAI (test)	Wanda + SparseSwaps	Accuracy64.7	60	1mo ago
Classification Suite Zero-shot	LLaMA-7B	Average Accuracy (Zero-Shot Suite)68.59	51	2d ago
5 zero-shot tasks		Accuracy79.48	43	1mo ago
Accuracy Benchmarks (PIQA, HellaSwag, LAMBADA, ARC-e, ARC-c, SciQ, Race, MMLU) Zero-shot		PIQA77.7	39	1mo ago
CUB 2011 (test)	ReViSE	Top-1 Accuracy68.1	34	1mo ago
WinoGrande, PiQA, HellaSwag, ARC-easy, ARC-challenge, BoolQ Zero-shot		Avg Zero-shot Acc76.88	31	1mo ago
HellaSwag, PIQA, COPA, ARC-C, ARC-E, WinoGrande		PIQA Acc80.79	30	1mo ago
ARC-Challenge, ARC-Easy, PIQA, WinoGrande Average		Accuracy73.9	27	17d ago
StanfordCars	FARE	Top-1 Clean Acc70.7	21	18d ago
STL10	TTE	Top-1 Clean Acc97.6	21	18d ago
EuroSAT		Top-1 Clean Accuracy68.5	21	18d ago
COCO-80		Accuracy (%)98	20	18d ago
Food 101	CLIPFT + A	Accuracy (Zero-shot)95.08	20	1mo ago
Flowers102	CLIP-PZSL	Top-1 Clean Acc89.86	19	1mo ago
Classification Datasets (MMLU, OBQA, ARC-e, WinoGrande, ARC-c, PIQA, HellaSwag)	LLAMA-7B	MMLU (5-shot)37.1	18	1mo ago
AwA 10-way 0-shot conventional setting	DEM	Hit@1 Accuracy86.7	18	1mo ago
OxfordPets	CLIP	Top-1 Clean Acc88.9	17	1mo ago
CUB 50-way 0-shot conventional setting	RELATION NET	Top-1 Accuracy62	16	1mo ago
Caltech101	TTE	Top-1 Clean Acc87.2	15	1mo ago
SUN	ACR	U Score60	14	1mo ago
Evaluation Suite Zero-shot (PiQA, LAMBDA, ARC-e, ARC-c, HellaS)	Longformer + SFA (k=8)	Decode Latency5.23	12	24d ago

Showing 25 of 72 rows