Share your thoughts, 1 month free Claude Pro on usSee more

Zero-shot Classification

Benchmarks

Dataset Name	SOTA Method	Metric
16 datasets average	CLIP	Zero-shot Accuracy (%)69.8	90	2mo ago
Downstream Tasks Zero-shot (BoolQ, HellaSwag, WinoGrande, ARC-e, ARC-c, PIQA, OBQA)		BoolQ Accuracy81.65	87	1mo ago
CIFAR100	CLIP-PZSL	Top-1 Accuracy74.83	65	14d ago
CIFAR10		Top-1 Clean Acc91.8	62	14d ago
EleutherAI (test)	Wanda + SparseSwaps	Accuracy64.7	60	3mo ago
5 zero-shot tasks		Accuracy79.48	55	2d ago
Classification Suite Zero-shot	LLaMA-7B	Average Accuracy (Zero-Shot Suite)68.59	51	1mo ago
Accuracy Benchmarks (PIQA, HellaSwag, LAMBADA, ARC-e, ARC-c, SciQ, Race, MMLU) Zero-shot		PIQA77.7	39	3mo ago
CUB 2011 (test)	ReViSE	Top-1 Accuracy68.1	34	3mo ago
WinoGrande, PiQA, HellaSwag, ARC-easy, ARC-challenge, BoolQ Zero-shot		Avg Zero-shot Acc76.88	31	3mo ago
Evaluation Suite (ARC-e, ARC-c, PIQA, HellaS., WinoG., BoolQ) Zero-shot (test)	AQLM	ARC-e Accuracy92.59	30	15d ago
Eight downstream tasks zero-shot		Accuracy (Zero-shot)47.7	30	1mo ago
HellaSwag, PIQA, COPA, ARC-C, ARC-E, WinoGrande		PIQA Acc80.79	30	3mo ago
ARC-Challenge, ARC-Easy, PIQA, WinoGrande Average		Accuracy73.9	27	2mo ago
StanfordCars	FARE	Top-1 Clean Acc70.7	21	14d ago
STL10	TTE	Top-1 Clean Acc97.6	21	14d ago
EuroSAT		Top-1 Clean Accuracy68.5	21	15d ago
20-task CLIP evaluation suite (test)	WUDI-Merging w/ E-PMQ	SUN Accuracy76.61	20	15d ago
COCO-80		Accuracy (%)98	20	2mo ago
ImageNet	R-TPT	Top-1 Clean Accuracy0.667	20	15d ago
Food 101	CLIPFT + A	Accuracy (Zero-shot)95.08	20	14d ago
Flowers102	CLIP-PZSL	Top-1 Clean Acc89.86	19	2mo ago
Classification Datasets (MMLU, OBQA, ARC-e, WinoGrande, ARC-c, PIQA, HellaSwag)	LLAMA-7B	MMLU (5-shot)37.1	18	3mo ago
ModelNet40	E-3DSNN-H	Accuracy85.4	18	15d ago
AwA 10-way 0-shot conventional setting	DEM	Hit@1 Accuracy86.7	18	3mo ago

Showing 25 of 87 rows