Dynamic 2D Gaussians: Geometrically Accurate Radiance Fields for Dynamic Objects
About
Reconstructing objects and extracting high-quality surfaces play a vital role in the real world. Current 4D representations show the ability to render high-quality novel views for dynamic objects, but cannot reconstruct high-quality meshes due to their implicit or geometrically inaccurate representations. In this paper, we propose a novel representation that can reconstruct accurate meshes from sparse image input, named Dynamic 2D Gaussians (D-2DGS). We adopt 2D Gaussians for basic geometry representation and use sparse-controlled points to capture the 2D Gaussian's deformation. By extracting the object mask from the rendered high-quality image and masking the rendered depth map, we remove floaters that are prone to occur during reconstruction and can extract high-quality dynamic mesh sequences of dynamic objects. Experiments demonstrate that our D-2DGS is outstanding in reconstructing detailed and smooth high-quality meshes from sparse inputs. The code is available at https://github.com/hustvl/Dynamic-2DGS.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Dynamic Surface Reconstruction | CMU Panoptic (Pizza1) | Accuracy12.6 | 12 | |
| Dynamic Surface Reconstruction | CMU Panoptic (Ian3) | Accuracy9.5 | 12 | |
| Dynamic Surface Reconstruction | CMU Panoptic Haggling b2 | Accuracy9.3 | 12 | |
| Dynamic Surface Reconstruction | CMU Panoptic (Band1) | Accuracy12.1 | 12 | |
| Dynamic Surface Reconstruction | Hi4D Cheers37 | Accuracy2.62 | 8 | |
| Dynamic Surface Reconstruction | Hi4D (Basketball13) | Accuracy3.55 | 8 | |
| Dynamic Surface Reconstruction | Hi4D (Talk22) | Accuracy2.75 | 8 | |
| Dynamic Surface Reconstruction | Hi4D (Fight17) | Accuracy2.33 | 8 | |
| Dynamic Surface Reconstruction | Hi4D (Football18) | Accuracy1.76 | 8 | |
| Dynamic Surface Reconstruction | Hi4D Backhug02 | Accuracy1.59 | 8 |