FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation
About
This technical report summarizes the winning solution for the 3D Occupancy Prediction Challenge, which is held in conjunction with the CVPR 2023 Workshop on End-to-End Autonomous Driving and CVPR 23 Workshop on Vision-Centric Autonomous Driving Workshop. Our proposed solution FB-OCC builds upon FB-BEV, a cutting-edge camera-based bird's-eye view perception design using forward-backward projection. On top of FB-BEV, we further study novel designs and optimization tailored to the 3D occupancy prediction task, including joint depth-semantic pre-training, joint voxel-BEV representation, model scaling up, and effective post-processing strategies. These designs and optimization result in a state-of-the-art mIoU score of 54.19% on the nuScenes dataset, ranking the 1st place in the challenge track. Code and models will be released at: https://github.com/NVlabs/FB-BEV.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| 3D Occupancy Prediction | Occ3D-nuScenes (val) | mIoU52.79 | 213 | |
| 3D Semantic Occupancy Prediction | SurroundOcc-nuScenes (val) | mIoU19.6 | 59 | |
| 3D Semantic Occupancy Prediction | Occ3D | RayIoU39.8 | 40 | |
| Semantic Occupancy Prediction | Occ3D (val) | mIoU42.1 | 37 | |
| 3D Semantic Occupancy Prediction | SurroundOcc (val) | mIoU19.6 | 36 | |
| Occupancy Prediction | Occ3D v1.0 (test) | RayIoU (Default)39 | 24 | |
| 3D Occupancy and Occupancy Flow | OpenOcc (val) | OccScore39.2 | 10 | |
| 3D Occupancy Prediction | Occ3D CVPR2023 Occupancy Challenge | mIoU40.7 | 6 | |
| 3D Scene Reconstruction | nuScenes (val) | Abs Rel0.342 | 5 |