Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video

About

Reconstructing high-fidelity, animatable 3D head avatars from effortlessly captured monocular videos is a pivotal yet formidable challenge. Although significant progress has been made in rendering performance and manipulation capabilities, notable challenges remain, including incomplete reconstruction and inefficient Gaussian representation. To address these challenges, we introduce FATE, a novel method for reconstructing an editable full-head avatar from a single monocular video. FATE integrates a sampling-based densification strategy to ensure optimal positional distribution of points, improving rendering efficiency. A neural baking technique is introduced to convert discrete Gaussian representations into continuous attribute maps, facilitating intuitive appearance editing. Furthermore, we propose a universal completion framework to recover non-frontal appearance, culminating in a 360$^\circ$-renderable 3D head avatar. FATE outperforms previous approaches in both qualitative and quantitative evaluations, achieving state-of-the-art performance. To the best of our knowledge, FATE is the first animatable and 360$^\circ$ full-head monocular reconstruction method for a 3D head avatar.

Jiawei Zhang, Zijian Wu, Zhiyang Liang, Yicheng Gong, Dongfang Hu, Yao Yao, Xun Cao, Hao Zhu• 2024

Related benchmarks

TaskDatasetResultRank
Head Avatar ReconstructionINSTA Dataset
PSNR28.33
14
3D Head Avatar ReconstructionNHA, NerFace, PointAvatar, INSTA, and custom captures (test)
L1 Error0.012
8
Monocular 3D Head Avatar CreationNeRSemble
PSNR19.1
8
Head Avatar ReconstructionINSTA dataset (test)--
8
Head Avatar ReconstructionPointAvatar Dataset
PSNR28.36
7
Head Avatar ReconstructionHDTF Dataset
PSNR27.18
7
Head Avatar ReconstructionNerFace Dataset
PSNR27.12
7
Monocular Facial Avatar ReconstructionOverall Combined Datasets (test)
PSNR28.37
6
Monocular Facial Avatar ReconstructionPointAvatar Dataset (test)
PSNR28.74
6
Monocular Facial Avatar ReconstructionOurs Dataset (test)
PSNR26.25
6
Showing 10 of 19 rows

Other info

Follow for update