Dynamic 2D Gaussians: Geometrically Accurate Radiance Fields for Dynamic Objects
About
Reconstructing objects and extracting high-quality surfaces play a vital role in the real world. Current 4D representations show the ability to render high-quality novel views for dynamic objects, but cannot reconstruct high-quality meshes due to their implicit or geometrically inaccurate representations. In this paper, we propose a novel representation that can reconstruct accurate meshes from sparse image input, named Dynamic 2D Gaussians (D-2DGS). We adopt 2D Gaussians for basic geometry representation and use sparse-controlled points to capture the 2D Gaussian's deformation. By extracting the object mask from the rendered high-quality image and masking the rendered depth map, we remove floaters that are prone to occur during reconstruction and can extract high-quality dynamic mesh sequences of dynamic objects. Experiments demonstrate that our D-2DGS is outstanding in reconstructing detailed and smooth high-quality meshes from sparse inputs. The code is available at https://github.com/hustvl/Dynamic-2DGS.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Novel View Synthesis | D-NeRF Hellwarrior | PSNR25.5 | 24 | |
| Novel View Synthesis | D-NeRF Trex | PSNR28.68 | 24 | |
| Novel View Synthesis | D-NeRF Jumpingjacks | PSNR29.29 | 18 | |
| Novel View Synthesis | D-NeRF BouncingBalls | PSNR27.78 | 18 | |
| Novel View Synthesis | D-NeRF Hook | PSNR27.8 | 18 | |
| Novel View Synthesis | D-NeRF Mutant | PSNR28.12 | 18 | |
| Novel View Synthesis | D-NeRF Standup | PSNR29.51 | 18 | |
| Novel View Synthesis | D-NeRF Lego | PSNR23.29 | 16 | |
| Novel View Synthesis | D-NeRF | PSNR25.82 | 16 | |
| Dynamic Surface Reconstruction | CMU Panoptic (Pizza1) | Accuracy12.6 | 12 |