Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Benchmarking and Analyzing Point Cloud Classification under Corruptions

About

3D perception, especially point cloud classification, has achieved substantial progress. However, in real-world deployment, point cloud corruptions are inevitable due to the scene complexity, sensor inaccuracy, and processing imprecision. In this work, we aim to rigorously benchmark and analyze point cloud classification under corruptions. To conduct a systematic investigation, we first provide a taxonomy of common 3D corruptions and identify the atomic corruptions. Then, we perform a comprehensive evaluation on a wide range of representative point cloud models to understand their robustness and generalizability. Our benchmark results show that although point cloud classification performance improves over time, the state-of-the-art methods are on the verge of being less robust. Based on the obtained observations, we propose several effective techniques to enhance point cloud classifier robustness. We hope our comprehensive benchmark, in-depth analysis, and proposed techniques could spark future research in robust 3D perception.

Jiawei Ren, Liang Pan, Ziwei Liu• 2022

Related benchmarks

TaskDatasetResultRank
3D Point Cloud ClassificationModelNet40 (test)
OA93.2
297
Point Cloud ClassificationScanObjectNN PB_T50_RS (test)
Overall Accuracy74.7
91
Point Cloud ClassificationModelNet-C (test)
mCE0.59
58
3D Point Cloud ClassificationModelNet40-C (test)
Robustness to Jitter Corruption0.876
25
ClassificationScanObjectNN-C
RmCE132.6
24
Point Cloud ClassificationModelNet40 (test)
mCE (Error Rate)0.863
17
Point Cloud ClassificationModelNet-C Unaugmented (test)
Overall Accuracy93
15
ClassificationModelNet40-C (test)
RmCE Error77.8
11
Point Cloud ClassificationModelNet-C Augmented WolfMix
Overall Accuracy (OA)93.3
8
ClassificationScanObjectNN-C 1.0 (test)
mOA52.2
8
Showing 10 of 10 rows

Other info

Code

Follow for update