Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

PPI++: Efficient Prediction-Powered Inference

About

We present PPI++: a computationally lightweight methodology for estimation and inference based on a small labeled dataset and a typically much larger dataset of machine-learning predictions. The methods automatically adapt to the quality of available predictions, yielding easy-to-compute confidence sets -- for parameters of any dimensionality -- that always improve on classical intervals using only the labeled data. PPI++ builds on prediction-powered inference (PPI), which targets the same problem setting, improving its computational and statistical efficiency. Real and synthetic experiments demonstrate the benefits of the proposed adaptations.

Anastasios N. Angelopoulos, John C. Duchi, Tijana Zrnic• 2023

Related benchmarks

TaskDatasetResultRank
Population property estimationDICES
Bias (MAE)0.06
92
LLM evaluation human preferencePPE Human Preference track
MSE / PPI0.283
28
LLM evaluation correctnessPPE Correctness track
MSE / PPI0.276
20
Bias Reduction EstimationPrivate Healthcare Census Setting
Average MAPE Difference-15.22
15
LLM win-rate estimation rankingLLM benchmark (Appendix)
Spearman Correlation1
14
Prediction Interval CoverageMalaria (MAR)
Coverage85
12
Politeness RatingPOPQUORN Avg Age
Coverage100
10
Population property estimationPRISM
Bias (MAE)1.03
8
Population property estimationSynthetic
Bias (MAE)0.69
8
Prediction Interval CoverageForest MCAR
Coverage89.4
6
Showing 10 of 23 rows

Other info

Follow for update