Model Selection

Benchmarks

Dataset Name	SOTA Method	Metric
DTD	PED	Weighted Kendall's Tau0.907	46	4d ago
SUN397	MODEL SPIDER	Weighted Kendall's Tau0.954	36	4d ago
Pets	LEAD	Weighted Kendall's Tau0.841	36	4d ago
CIFAR10	MODEL SPIDER	Weighted Kendall's Tau0.909	36	4d ago
CIFAR100	MODEL SPIDER	Weighted Kendall's Tau1	36	4d ago
Cars	MODEL SPIDER	Weighted Kendall's Tau0.785	36	4d ago
Caltech	LEAD	Weighted Kendall's Tau0.78	24	4d ago
VOC 2007	LEAD	Weighted Kendall's Tau0.743	17	4d ago
Flowers	LEAD	Weighted Kendall's Tau0.786	17	4d ago
Food	LEAD	Weighted Kendall's Tau0.892	17	4d ago
LOVM VLM Zoo original	SWAB	R_S Score0.498	8	4d ago
LOVM average over 23 datasets	SWAB	R5 Score0.504	8	4d ago
20 series benchmark pairs (test)	NEX	Pearson r0.778	4	4d ago
HuggingGPT Human Evaluation Set 130 diverse requests (test)	HuggingGPT	Passing Rate93.89	1	4d ago
MAPS (unseen)	Proposed Model Selection (PID-based)	Performance100	1	4d ago
MUSTARD (unseen)	Proposed Model Selection (PID-based)	Performance95.15	1	4d ago
MOSEI (unseen)	Proposed Model Selection (PID-based)	Performance99.35	1	4d ago
UR-FUNNY (unseen)	Proposed Model Selection (PID-based)	Performance98.58	1	4d ago
ENRICO (unseen)	Proposed Model Selection (PID-based)	Performance1	1	4d ago
MIMIC (unseen)	Proposed Model Selection (PID-based)	Performance99.78	1	4d ago
5 Synthetic Datasets (unseen)	Proposed Model Selection (PID-based)	Performance0.9991	1	4d ago

Showing 21 of 21 rows