360MonoDepth: High-Resolution 360{\deg} Monocular Depth Estimation
About
360{\deg} cameras can capture complete environments in a single shot, which makes 360{\deg} imagery alluring in many computer vision tasks. However, monocular depth estimation remains a challenge for 360{\deg} data, particularly for high resolutions like 2K (2048x1024) and beyond that are important for novel-view synthesis and virtual reality applications. Current CNN-based methods do not support such high resolutions due to limited GPU memory. In this work, we propose a flexible framework for monocular depth estimation from high-resolution 360{\deg} images using tangent images. We project the 360{\deg} input image onto a set of tangent planes that produce perspective views, which are suitable for the latest, most accurate state-of-the-art perspective monocular depth estimators. To achieve globally consistent disparity estimates, we recombine the individual depth estimates using deformable multi-scale alignment followed by gradient-domain blending. The result is a dense, high-resolution 360{\deg} depth map with a high level of detail, also for outdoor scenes which are not supported by existing methods. Our source code and data are available at https://manurare.github.io/360monodepth/.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Monocular Depth Estimation | Stanford2D3D (test) | δ1 Accuracy63.6 | 81 | |
| Monocular 360 Depth Estimation | Matterport3D official (test) | Delta Acc (1.25x)61.2 | 20 | |
| Panoramic Depth Estimation | Replica360 2K (test) | Absolute Relative Error0.167 | 12 | |
| Panoramic Depth Estimation | Matterport3D (test) | Abs Rel0.264 | 12 | |
| Monocular Depth Estimation | Replica360 2K resolution (2048x1024) (test) | AbsRel0.167 | 9 | |
| Monocular Depth Estimation | Matterport3D 2K resolution (2048x1024) (test) | AbsRel0.208 | 9 | |
| Monocular Depth Estimation | Replica360 4K (test) | AbsRel0.15 | 7 |