Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

About

This work presents Depth Anything, a highly practical solution for robust monocular depth estimation. Without pursuing novel technical modules, we aim to build a simple yet powerful foundation model dealing with any images under any circumstances. To this end, we scale up the dataset by designing a data engine to collect and automatically annotate large-scale unlabeled data (~62M), which significantly enlarges the data coverage and thus is able to reduce the generalization error. We investigate two simple yet effective strategies that make data scaling-up promising. First, a more challenging optimization target is created by leveraging data augmentation tools. It compels the model to actively seek extra visual knowledge and acquire robust representations. Second, an auxiliary supervision is developed to enforce the model to inherit rich semantic priors from pre-trained encoders. We evaluate its zero-shot capabilities extensively, including six public datasets and randomly captured photos. It demonstrates impressive generalization ability. Further, through fine-tuning it with metric depth information from NYUv2 and KITTI, new SOTAs are set. Our better depth model also results in a better depth-conditioned ControlNet. Our models are released at https://github.com/LiheYoung/Depth-Anything.

Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, Hengshuang Zhao• 2024

Related benchmarks

TaskDatasetResultRank
Semantic segmentationADE20K
mIoU59.4
936
Monocular Depth EstimationKITTI (Eigen)
Abs Rel0.046
502
Depth EstimationNYU v2 (test)
Threshold Accuracy (delta < 1.25)98.4
423
Monocular Depth EstimationNYU v2 (test)
Abs Rel4.3
257
Monocular Depth EstimationKITTI (Eigen split)
Abs Rel0.076
193
Depth CompletionNYU-depth-v2 official (test)--
187
Depth EstimationNYU Depth V2
RMSE0.206
177
Monocular Depth EstimationKITTI
Abs Rel0.074
161
Monocular Depth EstimationDDAD (test)
RMSE9.475
122
Monocular Depth EstimationETH3D
AbsRel1.682
117
Showing 10 of 133 rows
...

Other info

Code

Follow for update