Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets

About

Access to multiple predictive models trained for the same task, whether in regression or classification, is increasingly common in many applications. Aggregating their predictive uncertainties to produce reliable and efficient uncertainty quantification is therefore a critical but still underexplored challenge, especially within the framework of conformal prediction (CP). While CP methods can generate individual prediction sets from each model, combining them into a single, more informative set remains a challenging problem. To address this, we propose SACP (Symmetric Aggregated Conformal Prediction), a novel method that aggregates nonconformity scores from multiple predictors. SACP transforms these scores into e-values and combines them using any symmetric aggregation function. This flexible design enables a robust, data-driven framework for selecting aggregation strategies that yield sharper prediction sets. We also provide theoretical insights that help justify the validity and performance of the SACP approach. Extensive experiments on diverse datasets show that SACP consistently improves efficiency and often outperforms state-of-the-art model aggregation baselines.

Nabil Alami, Jad Zakharia, Souhaib Ben Taieb• 2025

Related benchmarks

TaskDatasetResultRank
Conformal PredictionCIFAR-10 (test)--
21
RegressionOpenML 361249
Coverage96.3
12
RegressionOpenML 361235
Coverage95.9
12
RegressionOpenML 361244
Coverage96.8
12
RegressionOpenML 361243
Coverage95.8
12
RegressionOpenML 361247
Coverage95
12
Conformal PredictionMNIST
Coverage (alpha=0.025)97.5
11
RegressionOpenML dataset 361234
Coverage95.5
6
RegressionOpenML 361236
Coverage94.9
6
RegressionOpenML 361237
Coverage97.8
6
Showing 10 of 15 rows

Other info

Follow for update