Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

CenterFormer: Center-based Transformer for 3D Object Detection

About

Query-based transformer has shown great potential in constructing long-range attention in many image-domain tasks, but has rarely been considered in LiDAR-based 3D object detection due to the overwhelming size of the point cloud data. In this paper, we propose CenterFormer, a center-based transformer network for 3D object detection. CenterFormer first uses a center heatmap to select center candidates on top of a standard voxel-based point cloud encoder. It then uses the feature of the center candidate as the query embedding in the transformer. To further aggregate features from multiple frames, we design an approach to fuse features through cross-attention. Lastly, regression heads are added to predict the bounding box on the output center feature representation. Our design reduces the convergence difficulty and computational complexity of the transformer structure. The results show significant improvements over the strong baseline of anchor-free object detection networks. CenterFormer achieves state-of-the-art performance for a single model on the Waymo Open Dataset, with 73.7% mAPH on the validation set and 75.6% mAPH on the test set, significantly outperforming all previously published CNN and transformer-based methods. Our code is publicly available at https://github.com/TuSimple/centerformer

Zixiang Zhou, Xiangchen Zhao, Yu Wang, Panqu Wang, Hassan Foroosh• 2022

Related benchmarks

TaskDatasetResultRank
3D Object DetectionWaymo Open Dataset (val)
3D APH Vehicle L273.8
175
3D Object DetectionWaymo Open Dataset (test)
Vehicle L2 mAPH77.7
105
3D Object DetectionWaymo Open Dataset (WOD) (val)
Vehicle L1 mAP78.8
47
3D Object DetectionWaymo Open Dataset LEVEL_2 (val)
3D AP (Overall)71.1
46
3D Object DetectionWaymo Open Dataset LEVEL_1 (val)--
46
3D Object DetectionWaymo Open 100% (val)
Vehicle AP (L1)78.1
36
3D Object DetectionWaymo Open Dataset 1.2 (val)
Vehicle mAP H L269.7
32
3D Object DetectionWaymo Open Dataset 0.2 labeled (val)
Vehicle 3D AP (L1)75.2
29
3D Object DetectionWaymo Open Dataset (WOD) (val)
Vehicle L1 3D AP78.8
27
3D Object DetectionWaymo Open Dataset v1.4 (val)
AP Vehicle (L1)75
23
Showing 10 of 15 rows

Other info

Code

Follow for update