Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

360MonoDepth: High-Resolution 360{\deg} Monocular Depth Estimation

About

360{\deg} cameras can capture complete environments in a single shot, which makes 360{\deg} imagery alluring in many computer vision tasks. However, monocular depth estimation remains a challenge for 360{\deg} data, particularly for high resolutions like 2K (2048x1024) and beyond that are important for novel-view synthesis and virtual reality applications. Current CNN-based methods do not support such high resolutions due to limited GPU memory. In this work, we propose a flexible framework for monocular depth estimation from high-resolution 360{\deg} images using tangent images. We project the 360{\deg} input image onto a set of tangent planes that produce perspective views, which are suitable for the latest, most accurate state-of-the-art perspective monocular depth estimators. To achieve globally consistent disparity estimates, we recombine the individual depth estimates using deformable multi-scale alignment followed by gradient-domain blending. The result is a dense, high-resolution 360{\deg} depth map with a high level of detail, also for outdoor scenes which are not supported by existing methods. Our source code and data are available at https://manurare.github.io/360monodepth/.

Manuel Rey-Area, Mingze Yuan, Christian Richardt• 2021

Related benchmarks

TaskDatasetResultRank
Monocular Depth EstimationStanford2D3D (test)
δ1 Accuracy63.6
81
Monocular 360 Depth EstimationMatterport3D official (test)
Delta Acc (1.25x)61.2
20
Panoramic Depth EstimationReplica360 2K (test)
Absolute Relative Error0.167
12
Panoramic Depth EstimationMatterport3D (test)
Abs Rel0.264
12
Monocular Depth EstimationReplica360 2K resolution (2048x1024) (test)
AbsRel0.167
9
Monocular Depth EstimationMatterport3D 2K resolution (2048x1024) (test)
AbsRel0.208
9
Monocular Depth EstimationReplica360 4K (test)
AbsRel0.15
7
Showing 7 of 7 rows

Other info

Code

Follow for update