Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videos

About

Dynamic 3D hand reconstruction from egocentric videos is essential for next-generation computing platforms such as AR/VR and AI glasses. Despite its importance, most prior works focus either on multi-view 3D hand reconstruction or on 4D human body reconstruction. Egocentric 4D hand reconstruction remains challenging due to fast head motion, rapid hand dynamics, severe occlusions, and inherent ambiguity from single-view observations. To address these challenges, we introduce Hand-4DGS, the first feed-forward framework for reconstructing dynamic 4D hands directly from egocentric videos, enabling both fast (~60 FPS) inference and strong generalization. Our approach incorporates a mesh-guided representation for structural priors and temporal convolutions to model dynamic motion. We evaluate our framework on two challenging egocentric datasets, H2O and ARCTIC, and demonstrate significant improvements over baselines. Our method benefits from the generalization capability of feed-forward networks and effective 2D image supervision through Gaussian splatting, without requiring expensive 3D hand pose ground-truth annotations.

Jeongmin Bae, Seoha Kim, Marc Pollefeys, Mahdi Rad, Youngjung Uh, Taein Kwon• 2026

Related benchmarks

TaskDatasetResultRank
3D Hand Pose EstimationH2O
MPJPE Both22.71
19
Hand ReconstructionH2O Viewpoint (train)
PSNR24.66
4
Hand ReconstructionARCTIC Viewpoint (train)
PSNR22.68
4
Novel View SynthesisH2O subject4_h1 sequences
PSNR18.14
4
Showing 4 of 4 rows

Other info

Follow for update