Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

XAGen: 3D Expressive Human Avatars Generation

About

Recent advances in 3D-aware GAN models have enabled the generation of realistic and controllable human body images. However, existing methods focus on the control of major body joints, neglecting the manipulation of expressive attributes, such as facial expressions, jaw poses, hand poses, and so on. In this work, we present XAGen, the first 3D generative model for human avatars capable of expressive control over body, face, and hands. To enhance the fidelity of small-scale regions like face and hands, we devise a multi-scale and multi-part 3D representation that models fine details. Based on this representation, we propose a multi-part rendering technique that disentangles the synthesis of body, face, and hands to ease model training and enhance geometric quality. Furthermore, we design multi-part discriminators that evaluate the quality of the generated avatars with respect to their appearance and fine-grained control capabilities. Experiments show that XAGen surpasses state-of-the-art methods in terms of realism, diversity, and expressive control abilities. Code and data will be made available at https://showlab.github.io/xagen.

Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Jiashi Feng, Mike Zheng Shou• 2023

Related benchmarks

TaskDatasetResultRank
Controllable Human Avatar GenerationDeepFashion
FID8.55
5
Controllable Human Avatar GenerationUBC
FID8.8
5
3D Human SynthesisDeepFashion
RGB Fidelity67.3
4
3D Human SynthesisMPV
RGB Score67.8
4
3D Human SynthesisUBC
RGB Fidelity57.2
4
3D Human SynthesisSHHQ
RGB Fidelity60.5
4
Controllable Human Avatar GenerationDeepFashion 36
Expression Error4.46
4
Controllable Human Avatar GenerationMPV 14
Exp Error6.31
4
Controllable Human Avatar GenerationMPV
FID7.94
4
Controllable Human Avatar GenerationSHHQ
FID5.88
4
Showing 10 of 12 rows

Other info

Code

Follow for update