Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DAS3R: Dynamics-Aware Gaussian Splatting for Static Scene Reconstruction

About

We propose a novel framework for scene decomposition and static background reconstruction from everyday videos. By integrating the trained motion masks and modeling the static scene as Gaussian splats with dynamics-aware optimization, our method achieves more accurate background reconstruction results than previous works. Our proposed method is termed DAS3R, an abbreviation for Dynamics-Aware Gaussian Splatting for Static Scene Reconstruction. Compared to existing methods, DAS3R is more robust in complex motion scenarios, capable of handling videos where dynamic objects occupy a significant portion of the scene, and does not require camera pose inputs or point cloud data from SLAM-based methods. We compared DAS3R against recent distractor-free approaches on the DAVIS and Sintel datasets; DAS3R demonstrates enhanced performance and robustness with a margin of more than 2 dB in PSNR. The project's webpage can be accessed via \url{https://kai422.github.io/DAS3R/}

Kai Xu, Tze Ho Elden Tse, Jizong Peng, Angela Yao• 2024

Related benchmarks

TaskDatasetResultRank
Video Object SegmentationDAVIS 2016
J-Measure41.13
50
Moving Object SegmentationDAVIS Moving 2016
Jaccard Index41.6
26
Video Object SegmentationDAVIS 17
J Score43.5
25
Point Cloud ReconstructionDyCheck
Accuracy (Mean)1.772
18
Dynamic SegmentationDyCheck
JM Score1.98
6
Pose EstimationDyCheck
ATE0.0189
6
Dynamic Occluded Region ReconstructionTrajectory-Match (Indoor-occluded)
PSNR16.4
4
Dynamic Occluded Region ReconstructionTrajectory-Match Street-occluded
PSNR20.18
4
Dynamic Occluded Region ReconstructionTrajectory-Match (Average-occluded)
PSNR16.93
4
Full Scene Static ReconstructionTrajectory-Match Indoor
PSNR21.45
4
Showing 10 of 14 rows

Other info

Follow for update