Weighted Aggregation of Conformity Scores for Classification
About
Conformal prediction is a powerful framework for constructing prediction sets with valid coverage guarantees in multi-class classification. However, existing methods often rely on a single score function, which can limit their efficiency and informativeness. We propose a novel approach that combines multiple score functions to improve the performance of conformal predictors by identifying optimal weights that minimize prediction set size. Our theoretical analysis establishes a connection between the weighted score functions and subgraph classes of functions studied in Vapnik-Chervonenkis theory, providing a rigorous mathematical basis for understanding the effectiveness of the proposed method. Experiments demonstrate that our approach consistently outperforms single-score conformal predictors while maintaining valid coverage, offering a principled and data-driven way to enhance the efficiency and practicality of conformal prediction in classification tasks.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Conformal Prediction | CIFAR-10 (test) | -- | 21 | |
| Regression | OpenML 361235 | Coverage97.2 | 12 | |
| Regression | OpenML 361243 | Coverage97.9 | 12 | |
| Regression | OpenML 361244 | Coverage97.1 | 12 | |
| Regression | OpenML 361247 | Coverage96.8 | 12 | |
| Regression | OpenML 361249 | Coverage96.1 | 12 | |
| Conformal Prediction | MNIST | Coverage (alpha=0.025)97.4 | 11 | |
| Regression | OpenML dataset 361234 | Coverage95.5 | 6 | |
| Regression | OpenML dataset 361237 | Coverage0.984 | 6 | |
| Regression | OpenML 361236 | Coverage95.2 | 6 |