Are We Ready for Radar to Replace Lidar in All-Weather Mapping and Localization?
About
We present an extensive comparison between three topometric localization systems: radar-only, lidar-only, and a cross-modal radar-to-lidar system across varying seasonal and weather conditions using the Boreas dataset. Contrary to our expectations, our experiments showed that our lidar-only pipeline achieved the best localization accuracy even during a snowstorm. Our results seem to suggest that the sensitivity of lidar localization to moderate precipitation has been exaggerated in prior works. However, our radar-only pipeline was able to achieve competitive accuracy with a much smaller map. Furthermore, radar localization and radar sensors still have room to improve and may yet prove valuable in extreme weather or as a redundant backup system. Code for this project can be found at: https://github.com/utiasASRL/vtr3
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Localization | Boreas (sequences) | Lateral RMSE (m)0.048 | 22 | |
| Odometry | Boreas (test) | Translational Error (%)1.97 | 8 | |
| LiDAR Localization | Boreas Road Trip (Suburban Route) | Lateral RMSE (m)0.024 | 4 | |
| LiDAR Localization | Boreas Road Trip (Industrial Route) | Lateral RMSE (m)0.021 | 4 | |
| LiDAR Localization | Boreas Road Trip (Regional Route) | Lateral RMSE (m)0.033 | 4 | |
| Radar Localization | Boreas Industrial | Longitude Error (m)0.08 | 4 | |
| SE(3) Odometry | Boreas-RT (suburbs) | Translation Error (%)29 | 4 | |
| SE(3) Odometry | Boreas-RT (industrial) | Translation Error (%)25 | 4 | |
| SE(3) Odometry | Boreas-RT (urban) | Translation Error (%)0.49 | 4 | |
| SE(3) Odometry | Boreas-RT (farm) | Translation Error (%)44 | 4 |