Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

RobustNeRF: Ignoring Distractors with Robust Losses

About

Neural radiance fields (NeRF) excel at synthesizing new views given multi-view, calibrated images of a static scene. When scenes include distractors, which are not persistent during image capture (moving objects, lighting variations, shadows), artifacts appear as view-dependent effects or 'floaters'. To cope with distractors, we advocate a form of robust estimation for NeRF training, modeling distractors in training data as outliers of an optimization problem. Our method successfully removes outliers from a scene and improves upon our baselines, on synthetic and real-world scenes. Our technique is simple to incorporate in modern NeRF frameworks, with few hyper-parameters. It does not assume a priori knowledge of the types of distractors, and is instead focused on the optimization problem rather than pre-processing or modeling transient objects. More results on our page https://robustnerf.github.io.

Sara Sabour, Suhani Vora, Daniel Duckworth, Ivan Krasin, David J. Fleet, Andrea Tagliasacchi• 2023

Related benchmarks

TaskDatasetResultRank
Novel View SynthesisRobustNeRF Android
PSNR24.38
37
Novel View SynthesisRobustNeRF Statue
PSNR21.56
37
Novel View SynthesisRobustNeRF Baby Yoda scene
LPIPS0.079
31
Novel View SynthesisRobustNeRF Crab
PSNR32.77
25
Novel View SynthesisNeRF On-the-go Medium Occlusion
PSNR21.72
18
Novel View SynthesisNeRF On-the-go High Occlusion
PSNR20.6
18
Novel View SynthesisNeRF On-the-go Low Occlusion
PSNR16.6
18
Novel View SynthesisOn-the-go Corner
LPIPS0.244
15
Novel View SynthesisOn-the-go (Patio)
LPIPS0.251
15
Novel View SynthesisOn-the-go (Spot)
LPIPS0.391
15
Showing 10 of 29 rows

Other info

Follow for update