Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits

About

Multimodal fusion requires integrating information from multiple sources that may conflict depending on context. Existing fusion approaches typically rely on static assumptions about source reliability, limiting their ability to resolve conflicts when a modality becomes unreliable due to situational factors such as sensor degradation or class-specific corruption. We introduce C$^2$MF, a context-specfic credibility-aware multimodal fusion framework that models per-instance source reliability using a Conditional Probabilistic Circuit (CPC). We formalize instance-level reliability through Context-Specific Information Credibility (CSIC), a KL-divergence-based measure computed exactly from the CPC. CSIC generalizes conventional static credibility estimates as a special case, enabling principled and adaptive reliability assessment. To evaluate robustness under cross-modal conflicts, we propose the Conflict benchmark, in which class-specific corruptions deliberately induce discrepancies between different modalities. Experimental results show that C$^2$MF improves predictive accuracy by up to 29% over static-reliability baselines in high-noise settings, while preserving the interpretability advantages of probabilistic circuit-based fusion.

Pranuthi Tenali, Sahil Sidheekh, Saurabh Mathur, Erik Blasch, Kristian Kersting, Sriraam Natarajan• 2026

Related benchmarks

TaskDatasetResultRank
Multimodal ClassificationConflict-AV-MNIST (test)
Accuracy99.61
32
Multimodal ClassificationConflict-NYUD (test)
Accuracy64.22
32
Multimodal ClassificationConflict-AV-MNIST 0% conflict
F1 Score99.22
8
Multimodal ClassificationConflict-AV-MNIST 50% conflict
F1-Score99.45
8
Multimodal ClassificationConflict-AV-MNIST 75% conflict
F1 Score99.52
8
Multimodal ClassificationConflict-AV-MNIST 100% conflict
F1-Score99.6
8
Multimodal ClassificationConflict-NYUD 0% conflict
F1 Score53.5
8
Multimodal ClassificationConflict-NYUD 50%
F1 Score53.27
8
Multimodal ClassificationNYUD 75% conflict
F1-Score55.46
8
Multimodal ClassificationConflict-NYUD 100% conflict
F1 Score58.53
8
Showing 10 of 20 rows

Other info

Follow for update