PaMoSplat: Part-Aware Motion-Guided Gaussian Splatting for Dynamic Scene Reconstruction
About
Dynamic scene reconstruction represents a fundamental yet demanding challenge in computer vision and robotics. While recent progress in 3DGS-based methods has advanced dynamic scene modeling, obtaining high-fidelity rendering and accurate tracking in scenarios with substantial, intricate motions remains significantly challenging. To address these challenges, we propose PaMoSplat, a novel dynamic Gaussian splatting framework incorporating part awareness and motion priors. Our approach is grounded in two key observations: 1) Parts serve as primitives for scene deformation, and 2) Motion cues from optical flow can effectively guide part motion. Specifically, PaMoSplat initializes by lifting multi-view segmentation masks into 3D space via graph clustering, establishing coherent Gaussian parts. For subsequent timestamps, we leverage a differential evolutionary algorithm to estimate the rigid motion of these parts using multi-view optical flow cues, providing a robust warm-start for further optimization. Additionally, PaMoSplat introduces an adaptive iteration count mechanism, internal learnable rigidity, and flow-supervised rendering loss to accelerate and optimize the training process. Comprehensive evaluations across diverse scenes, including real-world environments, demonstrate that PaMoSplat delivers superior rendering quality, improved tracking precision, and faster convergence compared to existing methods. Furthermore, it enables multiple part-level downstream applications, such as 4D scene editing.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Novel View Synthesis | Neural 3D Video Dataset Standard (All six scenes) | PSNR33.57 | 47 | |
| Novel View Synthesis | PanopticSports | PSNR29.83 | 16 | |
| Novel View Synthesis | self-captured dataset | Mean PSNR16.33 | 10 | |
| 3D Tracking | Self-captured dataset (apple_move) | MTE1.55 | 3 | |
| 3D Tracking | Self-captured dataset sponge_bowl | MTE2.21 | 3 | |
| 3D Tracking | Self-captured dataset (orange_bowl) | MTE1.85 | 3 | |
| 3D Tracking | Self-captured dataset app_pan_ora_bowl | MTE3.42 | 3 | |
| 3D Tracking | Self-captured dataset (building_block) | MTE2.79 | 3 | |
| 3D Tracking | self-captured dataset | MTE2.36 | 3 | |
| 3D Tracking | PanopticSports | Basketball MTE2.27 | 3 |