Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange

About

In the realm of point cloud scene understanding, particularly in indoor scenes, objects are arranged following human habits, resulting in objects of certain semantics being closely positioned and displaying notable inter-object correlations. This can create a tendency for neural networks to exploit these strong dependencies, bypassing the individual object patterns. To address this challenge, we introduce a novel self-supervised learning (SSL) strategy. Our approach leverages both object patterns and contextual cues to produce robust features. It begins with the formulation of an object-exchanging strategy, where pairs of objects with comparable sizes are exchanged across different scenes, effectively disentangling the strong contextual dependencies. Subsequently, we introduce a context-aware feature learning strategy, which encodes object patterns without relying on their specific context by aggregating object features across various scenes. Our extensive experiments demonstrate the superiority of our method over existing SSL techniques, further showing its better robustness to environmental changes. Moreover, we showcase the applicability of our approach by transferring pre-trained models to diverse point cloud datasets.

Yanhao Wu, Tong Zhang, Wei Ke, Congpei Qiu, Sabine Susstrunk, Mathieu Salzmann• 2024

Related benchmarks

TaskDatasetResultRank
Semantic segmentationS3DIS (Area 5)
mIOU66.9
799
Semantic segmentationScanNet V2 (val)
mIoU35.4
288
Semantic segmentationScanNet (val)
mIoU71.28
231
3D Visual GroundingScanRefer (val)--
155
Semantic segmentationScanNet
mIoU35.4
59
Instance SegmentationScanNet200 (val)
mAP@502.5
53
Instance SegmentationScanNet (val)--
39
Semantic segmentationScanNet200 v2 (val)
mIoU9.1
27
Semantic segmentationSynthia 4D (test)
mIoU77.48
26
Semantic segmentationSynth4D (val)
mIoU81.31
24
Showing 10 of 18 rows

Other info

Code

Follow for update