Representation Learning for Point Cloud Understanding
About
With the rapid advancement of technology, 3D data acquisition and utilization have become increasingly prevalent across various fields, including computer vision, robotics, and geospatial analysis. 3D data, captured through methods such as 3D scanners, LiDARs, and RGB-D cameras, provides rich geometric, shape, and scale information. When combined with 2D images, 3D data offers machines a comprehensive understanding of their environment, benefiting applications like autonomous driving, robotics, remote sensing, and medical treatment. This dissertation focuses on three main areas: supervised representation learning for point cloud primitive segmentation, self-supervised learning methods, and transfer learning from 2D to 3D. Our approach, which integrates pre-trained 2D models to support 3D network training, significantly improves 3D understanding without merely transforming 2D data. Extensive experiments validate the effectiveness of our methods, showcasing their potential to advance point cloud representation learning by effectively integrating 2D knowledge.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Semantic segmentation | S3DIS (Area 5) | mIOU73.8 | 799 | |
| 3D Object Detection | ScanNet V2 (val) | mAP@0.2576.1 | 352 | |
| Semantic segmentation | S3DIS (6-fold) | mIoU (Mean IoU)78.3 | 315 | |
| Shape classification | ModelNet40 (test) | -- | 255 | |
| Object Classification | ScanObjectNN OBJ_BG | Accuracy92.5 | 215 | |
| 3D Object Detection | ScanNet | mAP@0.2576.1 | 123 | |
| 3D Object Part Segmentation | ShapeNet Part (test) | -- | 114 | |
| 3D Object Detection | SUN RGB-D | mAP@0.2567.6 | 104 | |
| 3D Object Detection | SUN RGB-D v1 (val) | mAP@0.2567.6 | 81 | |
| 3D Object Classification | ScanObjectNN PB_T50_RS | OA91 | 72 |