Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Fully Explicit Dynamic Gaussian Splatting

About

3D Gaussian Splatting has shown fast and high-quality rendering results in static scenes by leveraging dense 3D prior and explicit representations. Unfortunately, the benefits of the prior and representation do not involve novel view synthesis for dynamic motions. Ironically, this is because the main barrier is the reliance on them, which requires increasing training and rendering times to account for dynamic motions. In this paper, we design a Explicit 4D Gaussian Splatting(Ex4DGS). Our key idea is to firstly separate static and dynamic Gaussians during training, and to explicitly sample positions and rotations of the dynamic Gaussians at sparse timestamps. The sampled positions and rotations are then interpolated to represent both spatially and temporally continuous motions of objects in dynamic scenes as well as reducing computational cost. Additionally, we introduce a progressive training scheme and a point-backtracking technique that improves Ex4DGS's convergence. We initially train Ex4DGS using short timestamps and progressively extend timestamps, which makes it work well with a few point clouds. The point-backtracking is used to quantify the cumulative error of each Gaussian over time, enabling the detection and removal of erroneous Gaussians in dynamic scenes. Comprehensive experiments on various scenes demonstrate the state-of-the-art rendering quality from our method, achieving fast rendering of 62 fps on a single 2080Ti GPU.

Junoh Lee, Chang-Yeon Won, Hyunjun Jung, Inhwan Bae, Hae-Gon Jeon• 2024

Related benchmarks

TaskDatasetResultRank
Novel View SynthesisNeural 3D Video Dataset Standard (All six scenes)
PSNR32.11
36
Dynamic Scene ReconstructionNeural 3D Video 19 (full)
PSNR32.11
17
Dynamic View SynthesisNeural 3D Video 19 (test)
PSNR32.11
16
Novel View SynthesisNeural 3D Video Dataset (Flame Salmon scene)
PSNR28.77
13
Novel View SynthesisTechnicolor
PSNR33.62
6
Novel View SynthesisNeural3DV
PSNR32.11
6
Novel View SynthesisENeRF-Outdoor
PSNR24.89
6
Novel View SynthesisSelfCap
PSNR24.96
6
Dyna mic Scen e Modelin gTechnicolor scene, frames #170-#269 (train)
PSNR32.24
4
Dynamic Scene RenderingTechnicolor Birthday scene, frames 50-169 (test)
PSNR30.56
4
Showing 10 of 10 rows

Other info

Follow for update