Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Lean Learning Beyond Clouds: Efficient Discrepancy-Conditioned Optical-SAR Fusion for Semantic Segmentation

About

Cloud occlusion severely degrades the semantic integrity of optical remote sensing imagery. While incorporating Synthetic Aperture Radar (SAR) provides complementary observations, achieving efficient global modeling and reliable cross-modal fusion under cloud interference remains challenging. Existing methods rely on dense global attention to capture long-range dependencies, yet such aggregation indiscriminately propagates cloud-induced noise. Improving robustness typically entails enlarging model capacity, which further increases computational overhead. Given the large-scale and high-resolution nature of remote sensing applications, such computational demands hinder practical deployment, leading to an efficiency-reliability trade-off. To address this dilemma, we propose EDC, an efficiency-oriented and discrepancy-conditioned optical-SAR semantic segmentation framework. A tri-stream encoder with Carrier Tokens enables compact global context modeling with reduced complexity. To prevent noise contamination, we introduce a Discrepancy-Conditioned Hybrid Fusion (DCHF) mechanism that selectively suppresses unreliable regions during global aggregation. In addition, an auxiliary cloud removal branch with teacher-guided distillation enhances semantic consistency under occlusion. Extensive experiments demonstrate that EDC achieves superior accuracy and efficiency, improving mIoU by 0.56\% and 0.88\% on M3M-CR and WHU-OPT-SAR, respectively, while reducing the number of parameters by 46.7\% and accelerating inference by 1.98$\times$. Our implementation is available at https://github.com/mengcx0209/EDC.

Chenxing Meng, Wuzhou Quan, Yingjie Cai, Liqun Cao, Liyan Zhang, Mingqiang Wei• 2026

Related benchmarks

TaskDatasetResultRank
Semantic segmentationWHU-OPT-SAR
mIoU49.58
15
Semantic segmentationM3M-CR Cloudy
mPA64.95
8
Semantic segmentationM3M-CR Cloud-Free
mPA71.05
8
Semantic segmentationM3M-CR (Overall)
mPA68.33
8
Semantic segmentationWHU-OPT-SAR (Cloudy)
Mean Pixel Accuracy (mPA)57.24
8
Semantic segmentationWHU-OPT-SAR Cloud-Free
mPA66.01
8
Semantic segmentationM3M-CR
mIoU54.2
7
Cloud RemovalM3M-CR (test)
SSIM91.85
6
Cloud RemovalWHU-OPT-SAR (test)
SSIM82.36
6
Semantic segmentationM3M-CR All Regions
ECE0.0046
5
Showing 10 of 12 rows

Other info

Follow for update