Learning to Defer

Benchmarks

Dataset Name	SOTA Method	Metric
CIFAR-100 Clean		Accuracy70.16	49	14d ago
CIFAR100 (test)	A-SM	Coverage98.9	36	1mo ago
CIFAR-10H (test)		Coverage100	25	1mo ago
CIFAR-10 with redundant synthetic experts	Decoupled	System Accuracy91.9	21	1mo ago
ImageNet varying-accuracy synthetic expert (val)	PiCCE-CE	Error41.18	20	3mo ago
CIFAR-100 varying-accuracy synthetic expert (test)	PiCCE-CE	Error Rate18.09	20	3mo ago
ImageNet Overlapped Dog Expert	PiCCE-CE	Error Rate41.23	20	3mo ago
ImageNet Dog Expert	PiCCE-CE	Error Rate41.32	20	3mo ago
CIFAR-100 Overlapped Animal Expert	PiCCE-CE	Error Rate15.1	20	3mo ago
CIFAR-100 Animal Expert	PiCCE-CE	Error Rate18.11	20	3mo ago
CIFAR-10H	Decoupled	System Accuracy96.1	18	1mo ago
Tiny ImageNet (test)		Deferral Ratio (%) - Expert 171.8	12	1mo ago
SVHN (test)		Deferral Ratio (Expert 1)83.27	12	1mo ago
CIFAR-10 (test)		Deferral Ratio (Expert 1)71.31	12	1mo ago
CIFAR-100	MILD	Deferral Loss0.4265	12	1mo ago
CIFAR-10	MILD	Deferral Loss0.106	12	1mo ago
ImageNet-16H ID	IFD	SAC76	12	3mo ago
Cifar100 Rapid Fatigue (test)	FALCON	AUACC72.36	10	2mo ago
Cifar100 Normal Fatigue (test)	FALCON	AUACC76.93	10	2mo ago
Cifar100 Sustained High Performance (test)	FALCON	AU Accuracy79.7	10	2mo ago
ImageNet 16H OOD	IFD	SAC76	9	3mo ago
Liver tumours ID	IFD	SAC88	8	3mo ago
Blood Cells ID	IFD	SAC89	8	3mo ago
HAM10000 ID	IFD	SAC86	8	3mo ago
SVHN	MILD	Deferral Loss0.0579	6	1mo ago

Showing 25 of 42 rows