Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FoundationStereo: Zero-Shot Stereo Matching

About

Tremendous progress has been made in deep stereo matching to excel on benchmark datasets through per-domain fine-tuning. However, achieving strong zero-shot generalization - a hallmark of foundation models in other computer vision tasks - remains challenging for stereo matching. We introduce FoundationStereo, a foundation model for stereo depth estimation designed to achieve strong zero-shot generalization. To this end, we first construct a large-scale (1M stereo pairs) synthetic training dataset featuring large diversity and high photorealism, followed by an automatic self-curation pipeline to remove ambiguous samples. We then design a number of network architecture components to enhance scalability, including a side-tuning feature backbone that adapts rich monocular priors from vision foundation models to mitigate the sim-to-real gap, and long-range context reasoning for effective cost volume filtering. Together, these components lead to strong robustness and accuracy across domains, establishing a new standard in zero-shot stereo depth estimation. Project page: https://nvlabs.github.io/FoundationStereo/

Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, Stan Birchfield• 2025

Related benchmarks

TaskDatasetResultRank
Stereo MatchingKITTI 2015 (test)
D1 Error (Overall)2.83
245
Stereo MatchingKITTI 2015
D1 Error (All)2.8
142
Stereo MatchingKITTI 2012--
108
Stereo MatchingKITTI 2012 (test)--
105
Stereo MatchingETH3D
bad 1.00.26
95
Stereo MatchingMiddlebury
Bad Pixel Rate (Thresh 2.0)1.1
84
Stereo MatchingMiddlebury (test)
EPE0.78
60
Stereo MatchingETH3D (non-occluded)
Bad 1.0 Error1.8
52
Stereo MatchingETH3D
Threshold Error > 1px (Noc)0.26
50
Stereo MatchingMiddlebury half resolution (test)
Threshold Error Rate1.12
36
Showing 10 of 65 rows

Other info

Follow for update