SOTA Zero-shot Classification benchmarks and papers with code

Benchmarks

Dataset Name	SOTA Method	Metric
16 datasets average	CLIP	Zero-shot Accuracy (%)69.8	90	4mo ago
Downstream Tasks Zero-shot (BoolQ, HellaSwag, WinoGrande, ARC-e, ARC-c, PIQA, OBQA)		BoolQ Accuracy81.65	87	3mo ago
CIFAR100	CLIP-PZSL	Top-1 Accuracy74.83	65	2mo ago
CIFAR10		Top-1 Clean Acc91.8	62	2mo ago
EleutherAI (test)	Wanda + SparseSwaps	Accuracy64.7	60	4mo ago
5 zero-shot tasks		Accuracy79.48	55	1mo ago
Classification Suite Zero-shot	LLaMA-7B	Average Accuracy (Zero-Shot Suite)68.59	51	3mo ago
Accuracy Benchmarks (PIQA, HellaSwag, LAMBADA, ARC-e, ARC-c, SciQ, Race, MMLU) Zero-shot		PIQA77.7	39	4mo ago
SUN	CV-DCLR	U Score62	34	18d ago
CUB 2011 (test)	ReViSE	Top-1 Accuracy68.1	34	4mo ago
WinoGrande, PiQA, HellaSwag, ARC-easy, ARC-challenge, BoolQ Zero-shot		Avg Zero-shot Acc76.88	31	4mo ago
Evaluation Suite (ARC-e, ARC-c, PIQA, HellaS., WinoG., BoolQ) Zero-shot (test)	AQLM	ARC-e Accuracy92.59	30	2mo ago
Eight downstream tasks zero-shot		Accuracy (Zero-shot)47.7	30	2mo ago
HellaSwag, PIQA, COPA, ARC-C, ARC-E, WinoGrande		PIQA Acc80.79	30	4mo ago
ARC-Challenge, ARC-Easy, PIQA, WinoGrande Average		Accuracy73.9	27	3mo ago
Caltech101	TTE	Top-1 Clean Acc87.2	24	1mo ago
StanfordCars	FARE	Top-1 Clean Acc70.7	21	2mo ago
STL10	TTE	Top-1 Clean Acc97.6	21	2mo ago
EuroSAT		Top-1 Clean Accuracy68.5	21	2mo ago
20-task CLIP evaluation suite (test)	WUDI-Merging w/ E-PMQ	SUN Accuracy76.61	20	2mo ago
COCO-80		Accuracy (%)98	20	3mo ago
ImageNet	R-TPT	Top-1 Clean Accuracy0.667	20	2mo ago
Food 101	CLIPFT + A	Accuracy (Zero-shot)95.08	20	2mo ago
Flowers102	CLIP-PZSL	Top-1 Clean Acc89.86	19	4mo ago
Classification Datasets (MMLU, OBQA, ARC-e, WinoGrande, ARC-c, PIQA, HellaSwag)	LLAMA-7B	MMLU (5-shot)37.1	18	4mo ago

Showing 25 of 97 rows