Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

FoundationStereo: Zero-Shot Stereo Matching

About

Tremendous progress has been made in deep stereo matching to excel on benchmark datasets through per-domain fine-tuning. However, achieving strong zero-shot generalization - a hallmark of foundation models in other computer vision tasks - remains challenging for stereo matching. We introduce FoundationStereo, a foundation model for stereo depth estimation designed to achieve strong zero-shot generalization. To this end, we first construct a large-scale (1M stereo pairs) synthetic training dataset featuring large diversity and high photorealism, followed by an automatic self-curation pipeline to remove ambiguous samples. We then design a number of network architecture components to enhance scalability, including a side-tuning feature backbone that adapts rich monocular priors from vision foundation models to mitigate the sim-to-real gap, and long-range context reasoning for effective cost volume filtering. Together, these components lead to strong robustness and accuracy across domains, establishing a new standard in zero-shot stereo depth estimation. Project page: https://nvlabs.github.io/FoundationStereo/

Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, Stan Birchfield• 2025

Related benchmarks

TaskDatasetResultRank
Stereo MatchingKITTI 2012 (test)--
76
Stereo MatchingETH3D
Threshold Error > 1px (All)0.48
30
Stereo MatchingBooster Q (test)
Error Rate (> 2%)5.18
26
Stereo Depth EstimationSQUID zero-shot
Relative Error (Rel)0.1095
16
Stereo MatchingDrivingStereo Zero-shot generalization
Error Rate (Sunny)3.22
15
Stereo Depth EstimationTartanAir underwater (test)
Relative Error (Rel)0.0542
13
3D ReconstructionShapeR Evaluation Dataset 1.0 (test)
CD6.483
10
Stereo Depth EstimationSUDS On Geometry Only
EPE1.89
8
DSM ReconstructionOmaha Diachronic DFC2019
Altitude MAE1.3
8
DSM ReconstructionJacksonville DFC2019
Altitude MAE (m)2.33
8
Showing 10 of 16 rows

Other info

Follow for update