Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Weighted Aggregation of Conformity Scores for Classification

About

Conformal prediction is a powerful framework for constructing prediction sets with valid coverage guarantees in multi-class classification. However, existing methods often rely on a single score function, which can limit their efficiency and informativeness. We propose a novel approach that combines multiple score functions to improve the performance of conformal predictors by identifying optimal weights that minimize prediction set size. Our theoretical analysis establishes a connection between the weighted score functions and subgraph classes of functions studied in Vapnik-Chervonenkis theory, providing a rigorous mathematical basis for understanding the effectiveness of the proposed method. Experiments demonstrate that our approach consistently outperforms single-score conformal predictors while maintaining valid coverage, offering a principled and data-driven way to enhance the efficiency and practicality of conformal prediction in classification tasks.

Rui Luo, Zhixin Zhou• 2024

Related benchmarks

TaskDatasetResultRank
Conformal PredictionCIFAR-10 (test)--
21
RegressionOpenML 361235
Coverage97.2
12
RegressionOpenML 361243
Coverage97.9
12
RegressionOpenML 361244
Coverage97.1
12
RegressionOpenML 361247
Coverage96.8
12
RegressionOpenML 361249
Coverage96.1
12
Conformal PredictionMNIST
Coverage (alpha=0.025)97.4
11
RegressionOpenML dataset 361234
Coverage95.5
6
RegressionOpenML dataset 361237
Coverage0.984
6
RegressionOpenML 361236
Coverage95.2
6
Showing 10 of 15 rows

Other info

Follow for update