Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DeSplat: Decomposed Gaussian Splatting for Distractor-Free Rendering

About

Gaussian splatting enables fast novel view synthesis in static 3D environments. However, reconstructing real-world environments remains challenging as distractors or occluders break the multi-view consistency assumption required for accurate 3D reconstruction. Most existing methods rely on external semantic information from pre-trained models, introducing additional computational overhead as pre-processing steps or during optimization. In this work, we propose a novel method, DeSplat, that directly separates distractors and static scene elements purely based on volume rendering of Gaussian primitives. We initialize Gaussians within each camera view for reconstructing the view-specific distractors to separately model the static 3D scene and distractors in the alpha compositing stages. DeSplat yields an explicit scene separation of static elements and distractors, achieving comparable results to prior distractor-free approaches without sacrificing rendering speed. We demonstrate DeSplat's effectiveness on three benchmark data sets for distractor-free novel view synthesis. See the project website at https://aaltoml.github.io/desplat/.

Yihao Wang, Marcus Klasson, Matias Turkulainen, Shuzhe Wang, Juho Kannala, Arno Solin• 2024

Related benchmarks

TaskDatasetResultRank
Novel View SynthesisNeRF On-the-go (test)
Corner Score16.89
18
Novel View SynthesisRobustNeRF
Android Quality Score18.71
18
Novel View SynthesisSacre Coeur Phototourism (test)
PSNR21.74
16
Novel View SynthesisTrevi Fountain Phototourism (test)
PSNR21.48
16
Novel View SynthesisNeRF On-the-go--
15
Novel View SynthesisBrandenburg Gate Phototourism (test)
PSNR26.58
11
Novel View SynthesisNeRF On-the-go Corner
PSNR26.18
8
Novel View SynthesisNeRF On-the-go Spot
PSNR24.96
8
Distractor-free scene reconstructionNeRF On-the-go 20 (hold-out views)
PSNR (Mountain)19.59
8
Novel View SynthesisNeRF On-the-go Patio-High
PSNR22.37
8
Showing 10 of 19 rows

Other info

Follow for update