Test-time Adaptive Hierarchical Co-enhanced Denoising Network for Reliable Multimodal Classification
About
Reliable learning of multimodal data (e.g., multi-omics) is a widely concerning issue, especially in safety-critical applications such as medical diagnosis. However, low-quality data induced by multimodal noise poses a major challenge in this domain, causing existing methods to suffer from two key limitations. First, they struggle to handle heterogeneous data noise, hindering robust multimodal representation learning. Second, they exhibit limited adaptability and generalization when encountering previously unseen noise. To address these issues, we propose Test-time Adaptive Hierarchical Co-enhanced Denoising Network (TAHCD). On one hand, TAHCD introduces the Adaptive Stable Subspace Alignment and Sample-Adaptive Confidence Alignment to reliably remove heterogeneous noise. They account for noise at both global and instance levels and enable jointly removal of modality-specific and cross-modality noise, achieving robust learning. On the other hand, TAHCD introduces Test-Time Cooperative Enhancement, which adaptively updates the model in response to input noise in a label-free manner, thus improving generalization. This is achieved by collaboratively enhancing the joint removal process of modality-specific and cross-modality noise across global and instance levels according to sample noise. Experiments on multiple benchmarks demonstrate that the proposed method achieves superior classification performance, robustness, and generalization compared with state-of-the-art reliable multimodal learning approaches.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Multimodal Classification | BRCA (train test) | Accuracy85.6 | 36 | |
| Multimodal Classification | ROSMAP (train test) | Accuracy87.7 | 36 | |
| Multimodal Classification | CUB (train test) | Accuracy0.938 | 36 | |
| Multimodal Classification | FOOD101 UPMC (train test) | Accuracy94.2 | 36 | |
| Classification | BRCA (test) | Accuracy85.6 | 15 | |
| Multimodal Classification | BRCA multimodal noise η=10%, ε=5 original (test) | Accuracy67.4 | 9 | |
| Multimodal Classification | ROSMAP multimodal noise η=10%, ε=5 original (test) | Acc75.1 | 9 | |
| Multimodal Classification | CUB multimodal noise η=10%, ε=5 original (test) | Accuracy72.4 | 9 | |
| Multimodal Classification | UPMC FOOD101 multimodal noise η=10%, ε=5 original (test) | Accuracy74.2 | 9 |