SOTA Uncertainty Calibration benchmarks and papers with code

Benchmarks

Dataset Name	SOTA Method	Metric
CIFAR-10-C	AUGMIX	RMS Calibration Error5.2	35	1mo ago
QUBIQ 2021	Tackle	Expected Calibration Error (ECE)0.81	28	3d ago
CIFAR-F	Knowledge-Transferring-based Temperature Scaling	ECE5.97	27	1mo ago
CIFAR-10.1 C	Knowledge-Transferring-based Temperature Scaling	ECE29.98	27	1mo ago
CIFAR-10.1	Knowledge-Transferring-based Temperature Scaling	ECE0.0395	27	1mo ago
RewardBench	Verbalized	Kuiper0.009	24	1mo ago
JudgeBench	Probe	Kuiper0.037	24	1mo ago
GPQA diamond	ET-PE	AUROC0.696	18	1mo ago
GPQA main	PE	AUROC0.672	18	1mo ago
SciBench	PE	AUROC78.7	18	1mo ago
MATH 500	ET-PE	AUROC0.861	18	1mo ago
OASIS	S+LC	AUSE0.0659	8	1mo ago
VTAB	CLAP (Ours) + CoOp	ECE13.6	8	1mo ago
CUB200	CLAP (Ours) + MaPLe	ECE18.4	8	1mo ago
ImageNet-R	CLAP (Ours) + MaPLe	ECE0.146	8	1mo ago
ImageNet 100	CLAP (Ours) + AttriCLIP	ECE0.205	8	1mo ago
CIFAR100	MaPLe	ECE0.168	8	1mo ago
Sentinel-1 Ice Pack	Bayesian Transformer	Weighted ECE0.17	6	1mo ago
Sentinel-1 MIZ	Bayesian Transformer	Weighted ECE0.18	6	1mo ago
MEPS Panel 21 2017 (test)	NRC-Embed	Length242	6	1mo ago
MEPS Panel 20 2017 (test)	DeepEnsemble	Length19,050	6	1mo ago
MEPS Panel 19 2017 (test)	NRC-Embed	Length240.5	6	1mo ago
Boston	NRC	MACE5	6	1mo ago
Concrete	NRC	MACE0.05	6	1mo ago
Auto-MPG MPG3	NRC	MACE0.08	6	1mo ago

Showing 25 of 47 rows