RobustNeRF: Ignoring Distractors with Robust Losses
About
Neural radiance fields (NeRF) excel at synthesizing new views given multi-view, calibrated images of a static scene. When scenes include distractors, which are not persistent during image capture (moving objects, lighting variations, shadows), artifacts appear as view-dependent effects or 'floaters'. To cope with distractors, we advocate a form of robust estimation for NeRF training, modeling distractors in training data as outliers of an optimization problem. Our method successfully removes outliers from a scene and improves upon our baselines, on synthetic and real-world scenes. Our technique is simple to incorporate in modern NeRF frameworks, with few hyper-parameters. It does not assume a priori knowledge of the types of distractors, and is instead focused on the optimization problem rather than pre-processing or modeling transient objects. More results on our page https://robustnerf.github.io.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Novel View Synthesis | RobustNeRF Baby Yoda scene | LPIPS0.079 | 20 | |
| Novel View Synthesis | RobustNeRF Android | PSNR24.38 | 17 | |
| Novel View Synthesis | RobustNeRF Statue | PSNR21.56 | 17 | |
| Novel View Synthesis | RobustNeRF Crab | PSNR32.77 | 16 | |
| Novel View Synthesis | On-the-go Dataset | PSNR (Mountain)21.37 | 12 | |
| Novel View Synthesis | RobustNeRF Avg. | PSNR28.11 | 12 | |
| View Synthesis | Synthetic Dataset average across three synthetic scenes | PSNR20.59 | 10 | |
| Novel View Synthesis | Real Dataset 6 outdoor scenes (test) | PSNR20.78 | 8 | |
| Novel View Synthesis | Kubric Car | LPIPS0.013 | 6 | |
| Novel View Synthesis | Kubric Bag | LPIPS0.006 | 6 |