SOTA Uncertainty Quantification benchmarks and papers with code

Benchmarks

Dataset Name	SOTA Method	Metric
Average of 6 datasets	Dissimilarity + beamsearch	PRR65	120	1mo ago
Musique 500 randomly sampled queries (test)	R2C	AUROC0.8322	70	1mo ago
HotpotQA 500 randomly sampled queries (test)	R2C	AUROC83.25	70	1mo ago
PopQA 500 randomly sampled queries (test)	R2C	AUROC0.8709	70	1mo ago
ImageNet 10	lowrank-KFAC	NLL0.266	42	5d ago
CIFAR10	lowrank-KFAC	NLL0.256	42	5d ago
FashionMNIST	lowrank-KFAC	NLL0.248	42	5d ago
Vision Datasets averaged (test)	SGPU	AUROC81.7	36	1mo ago
LongFact	Ecc	PCC-0.017	32	9d ago
BIO	Ecc	PCC-0.129	32	9d ago
MulFactTrap (test)	RUfact	ROC AUC0.898	32	1mo ago
Mixed Dataset (real and fake biographies)	RUgen	ROC AUC0.9001	32	1mo ago
MAQA ∆K−1	Structure-Aware Minimum Bayes Risk Decoding	KL Divergence AUC0.757	28	1mo ago
CNN/DailyMail	Structure-Aware Minimum Bayes Risk Decoding	Hamming AUC0.745	28	1mo ago
WMT 19	KLE	COMET AUC0.608	28	1mo ago
MAQA	Structure-Aware Minimum Bayes Risk Decoding	Hamming AUC83.5	28	1mo ago
SciQ (test)	SENTSAR	AUROC74.5	28	1mo ago
MSD Task01 (test)	ACQR	Coverage (%)94.22	24	15d ago
ARC-E	ScalaBL	Training Memory (MB)17,290	14	11d ago
CIFAR-10 (test)	MAP	Accuracy93.5	14	1mo ago
PTB		CU417	12	1mo ago
MIT-BIH		CU998	12	1mo ago
MedQA (test)	SAR	AUROC0.635	9	1mo ago
Countdown, GSM8K, MATH500, SVAMP Combined	DiSE	Average ROC-AUC63.7	8	1mo ago
MATH500	LLaMA perplexity	ROC-AUC (Threshold 128)65.2	8	1mo ago

Showing 25 of 66 rows