One-Shot Federated Conformal Prediction
About
In this paper, we introduce a conformal prediction method to construct prediction sets in a oneshot federated learning setting. More specifically, we define a quantile-of-quantiles estimator and prove that for any distribution, it is possible to output prediction sets with desired coverage in only one round of communication. To mitigate privacy issues, we also describe a locally differentially private version of our estimator. Finally, over a wide range of experiments, we show that our method returns prediction sets with coverage and length very similar to those obtained in a centralized setting. Overall, these results demonstrate that our method is particularly well-suited to perform conformal predictions in a one-shot federated learning setting.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Conformal Prediction | GAUSSIAN | Conditional Coverage100 | 16 | |
| Conformal Prediction | Airfoil | CMC (%)1.6 | 16 | |
| Conformal Prediction | Crime | CMC1.57 | 16 | |
| Conformal Prediction | Concrete | Conditional Marginal Coverage3.37 | 16 | |
| Conformal Prediction | BIKE | Empirical Coverage88.2 | 12 | |
| Conformal Prediction | CIFAR10C | Mean Coverage (MC)89.4 | 8 | |
| Conformal Prediction | PROTEIN | MC90.2 | 8 | |
| Uncertainty-bound prediction | Poisson | MC (%)89.97 | 8 | |
| Conformal Prediction | CIFAR-10-C | CCC62.4 | 8 | |
| Conformal Prediction | STAR | MC (%)88.78 | 8 |