Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather

About

LiDAR semantic segmentation (LSS) is a critical task in autonomous driving and has achieved promising progress. However, prior LSS methods are conventionally investigated and evaluated on datasets within the same domain in clear weather. The robustness of LSS models in unseen scenes and all weather conditions is crucial for ensuring safety and reliability in real applications. To this end, we propose UniMix, a universal method that enhances the adaptability and generalizability of LSS models. UniMix first leverages physically valid adverse weather simulation to construct a Bridge Domain, which serves to bridge the domain gap between the clear weather scenes and the adverse weather scenes. Then, a Universal Mixing operator is defined regarding spatial, intensity, and semantic distributions to create the intermediate domain with mixed samples from given domains. Integrating the proposed two techniques into a teacher-student framework, UniMix efficiently mitigates the domain gap and enables LSS models to learn weather-robust and domain-invariant representations. We devote UniMix to two main setups: 1) unsupervised domain adaption, adapting the model from the clear weather source domain to the adverse weather target domain; 2) domain generalization, learning a model that generalizes well to unseen scenes in adverse weather. Extensive experiments validate the effectiveness of UniMix across different tasks and datasets, all achieving superior performance over state-of-the-art methods. The code will be released.

Haimei Zhao, Jing Zhang, Zhuo Chen, Shanshan Zhao, Dacheng Tao• 2024

Related benchmarks

TaskDatasetResultRank
3D Point Cloud Semantic SegmentationSemanticSTF SemanticKITTI (val)
mIoU31.4
23
Semantic segmentationsemanticKITTI SynLiDAR source (val)
mIoU (Mean IoU)33.7
21
3D Semantic SegmentationSynLiDAR to SemanticSTF (val)
mIoU (D-fog)24.3
16
Semantic segmentationSemanticSTF SemanticKITTI 1.0 (test)
car IoU82.7
9
Semantic segmentationSemanticSTF SynLiDAR 1.0 (test)
Car IoU65.4
8
LiDAR Semantic SegmentationParis-Lille-3D
Ground IoU93.44
6
Semantic segmentationSTPLS3D to Toronto-3D (test)
Ground IoU95.85
6
Semantic segmentationSTPLS3D to DALES (test)
Ground IoU91.02
6
LiDAR Semantic SegmentationISPRS Vaihingen
Ground IoU0.7354
6
LiDAR Semantic SegmentationSemanticSTF (val)
mIoU37.2
4
Showing 10 of 10 rows

Other info

Follow for update