NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild
About
Neural Radiance Fields (NeRFs) have shown remarkable success in synthesizing photorealistic views from multi-view images of static scenes, but face challenges in dynamic, real-world environments with distractors like moving objects, shadows, and lighting changes. Existing methods manage controlled environments and low occlusion ratios but fall short in render quality, especially under high occlusion scenarios. In this paper, we introduce NeRF On-the-go, a simple yet effective approach that enables the robust synthesis of novel views in complex, in-the-wild scenes from only casually captured image sequences. Delving into uncertainty, our method not only efficiently eliminates distractors, even when they are predominant in captures, but also achieves a notably faster convergence speed. Through comprehensive experiments on various scenes, our method demonstrates a significant improvement over state-of-the-art techniques. This advancement opens new avenues for NeRF in diverse and dynamic real-world applications.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Novel View Synthesis | D-RE10K static regions only (test) | PSNR19.52 | 26 | |
| Novel View Synthesis | D-RE10K-iPhone full-image fidelity (test) | PSNR17.53 | 26 | |
| Novel View Synthesis | RobustNeRF Baby Yoda scene | LPIPS0.24 | 8 | |
| Novel View Synthesis | Kubric Cars | LPIPS0.035 | 6 | |
| Novel View Synthesis | RobustNeRF Android (test) | LPIPS0.21 | 6 | |
| Novel View Synthesis | On-the-go (Fountain) | LPIPS0.314 | 6 | |
| Novel View Synthesis | On-the-go Corner | LPIPS0.19 | 6 | |
| Novel View Synthesis | On-the-go (Patio) | LPIPS0.219 | 6 | |
| Novel View Synthesis | On-the-go (Spot) | LPIPS0.189 | 6 | |
| Novel View Synthesis | On-the-go Patio-High | LPIPS0.235 | 6 |