Learning to Defer

Benchmarks

Dataset Name	SOTA Method	Metric
CIFAR-100 Clean		Accuracy70.16	49	2mo ago
CIFAR100 (test)	A-SM	Coverage98.9	36	2mo ago
CIFAR-10H (test)		Coverage100	25	3mo ago
CIFAR-10 with redundant synthetic experts	Decoupled	System Accuracy91.9	21	3mo ago
ImageNet varying-accuracy synthetic expert (val)	PiCCE-CE	Error41.18	20	4mo ago
CIFAR-100 varying-accuracy synthetic expert (test)	PiCCE-CE	Error Rate18.09	20	4mo ago
ImageNet Overlapped Dog Expert	PiCCE-CE	Error Rate41.23	20	4mo ago
ImageNet Dog Expert	PiCCE-CE	Error Rate41.32	20	4mo ago
CIFAR-100 Overlapped Animal Expert	PiCCE-CE	Error Rate15.1	20	4mo ago
CIFAR-100 Animal Expert	PiCCE-CE	Error Rate18.11	20	4mo ago
CIFAR-10H	Decoupled	System Accuracy96.1	18	3mo ago
Tiny ImageNet (test)		Deferral Ratio (%) - Expert 171.8	12	2mo ago
SVHN (test)		Deferral Ratio (Expert 1)83.27	12	2mo ago
CIFAR-10 (test)		Deferral Ratio (Expert 1)71.31	12	2mo ago
CIFAR-100	MILD	Deferral Loss0.4265	12	2mo ago
CIFAR-10	MILD	Deferral Loss0.106	12	2mo ago
ImageNet-16H ID	IFD	SAC76	12	4mo ago
Cifar100 Rapid Fatigue (test)	FALCON	AUACC72.36	10	3mo ago
Cifar100 Normal Fatigue (test)	FALCON	AUACC76.93	10	3mo ago
Cifar100 Sustained High Performance (test)	FALCON	AU Accuracy79.7	10	3mo ago
ImageNet 16H OOD	IFD	SAC76	9	4mo ago
Liver tumours ID	IFD	SAC88	8	4mo ago
Blood Cells ID	IFD	SAC89	8	4mo ago
HAM10000 ID	IFD	SAC86	8	4mo ago
SVHN	MILD	Deferral Loss0.0579	6	2mo ago

Showing 25 of 42 rows