Zero-shot Classification

Benchmarks

Dataset Name	SOTA Method	Metric
EleutherAI (test)	Wanda + SparseSwaps	Accuracy64.7	60	4d ago
Accuracy Benchmarks (PIQA, HellaSwag, LAMBADA, ARC-e, ARC-c, SciQ, Race, MMLU) Zero-shot		PIQA77.7	39	4d ago
CUB 2011 (test)	ReViSE	Top-1 Accuracy68.1	34	4d ago
WinoGrande, PiQA, HellaSwag, ARC-easy, ARC-challenge, BoolQ Zero-shot		Avg Zero-shot Acc76.88	31	4d ago
HellaSwag, PIQA, COPA, ARC-C, ARC-E, WinoGrande		PIQA Acc80.79	30	4d ago
Classification Suite Zero-shot	LLaMA-7B	Average Accuracy (Zero-Shot Suite)68.59	29	4d ago
Classification Datasets (MMLU, OBQA, ARC-e, WinoGrande, ARC-c, PIQA, HellaSwag)	LLAMA-7B	MMLU (5-shot)37.1	18	4d ago
AwA 10-way 0-shot conventional setting	DEM	Hit@1 Accuracy86.7	18	4d ago
OxfordPets	CLIP	Top-1 Clean Acc88.9	17	2d ago
CUB 50-way 0-shot conventional setting	RELATION NET	Top-1 Accuracy62	16	4d ago
Caltech101	TTE	Top-1 Clean Acc87.2	15	2d ago
SUN	ACR	U Score60	14	4d ago
Food 101	CLIPFT + A	Accuracy (Zero-shot)95.08	11	2d ago
EuroSAT	CLIPFT + A	Accuracy81.82	11	2d ago
Fine-Grained datasets (OxfordPets, Flowers102, Food101, StanfordCars)	R-TPT	Top-1 Accuracy (Clean)76.7	10	2d ago
Zero-shot Evaluation Suite (AC, AE, WI, QA) v1	LoPRov-FT	AC Score46.2	10	4d ago
PCAM	R-TPT	Top-1 Clean Acc54.6	10	2d ago
FGVCAircraft	R-TPT	Top-1 Clean Acc23.8	10	2d ago
Country211	R-TPT	Top-1 Clean Acc19.2	10	2d ago
SUN397	TTE	Top-1 Clean Acc65.4	10	2d ago
StanfordCars	FARE	Top-1 Clean Acc70.7	10	2d ago
Flowers102	CLIP	Top-1 Clean Acc66	10	2d ago
Caltech256	R-TPT	Top-1 Clean Acc88	10	2d ago
STL10	TTE	Top-1 Clean Acc97.6	10	2d ago
CIFAR100		Top-1 Clean Accuracy63.4	10	2d ago

Showing 25 of 54 rows