SOTA Uncertainty Estimation benchmarks and papers with code

Benchmarks

Dataset Name	SOTA Method	Metric
TriviaQA	Geometry-Calibrated Conformal Abstention	AUROC88	111	1mo ago
TriviaQA (test)	ACT-ViT	AUROC87.91	110	2d ago
JudgeBench (test)	Energy	AUROC71.53	77	3mo ago
CoQA	Logit Magnitude	AUROC0.857	58	26d ago
SciQA	Eigenscore	AUROC0.8269	56	2mo ago
emrQA	Logit Magnitude	AUROC77.5	42	26d ago
NewsQA	Self-Consistency	AUROC76.6	42	26d ago
CoQA (test)	SAR	AUROC77.3	42	3mo ago
GSM8K	BSDETECTOR	AUROC0.951	41	1mo ago
TruthfulQA		AUROC63.9	40	1mo ago
NaturalQA	SAR	AUROC77	39	1mo ago
Real-Noise	Deep Ens.	Uncertainty Interval Length0.021	36	8d ago
Secret-word taboo dataset (random-split)	Bootstrap	Accuracy42.4	32	7d ago
NQ-open	SE+MCH	AUROC75.88	32	1mo ago
Gaussian noise dataset	Deep Ens.	Uncertainty Interval Length0.05	30	8d ago
Poisson noise dataset	QUTCC	Uncertainty Interval Length0.035	30	8d ago
WebQA	SE + MARS	AUROC73.57	30	3mo ago
OBQA	DeepEns	AUROC88.03	24	1mo ago
RACE Llama-3.1-8B and Gemma-2-9B backbones (test)	ETN	AUROC91.3	24	1mo ago
SimpleQA, MuSiQue, and TruthfulQA Average	Internal Confidence	AUROC61	24	3mo ago
MuSiQue	Predictive Entropy	AUROC65.6	24	3mo ago
SimpleQA	P(YES) (top right)	AUROC61.3	24	3mo ago
TriviaQA, SVAMP, and NQ Average	G-NLL	AUROC0.843	23	1mo ago
CommonsenseQA	Geometry-Calibrated Conformal Abstention	AUROC0.74	18	1mo ago
SciQ	Geometry-Calibrated Conformal Abstention	AUROC82	18	1mo ago

Showing 25 of 83 rows