Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

EmoTaG: Emotion-Aware Talking Head Synthesis on Gaussian Splatting with Few-Shot Personalization

About

Audio-driven 3D talking head synthesis has advanced rapidly with Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). By leveraging rich pre-trained priors, few-shot methods enable instant personalization from just a few seconds of video. However, under expressive facial motion, existing few-shot approaches often suffer from geometric instability and audio-emotion mismatch, highlighting the need for more effective emotion-aware motion modeling. In this work, we present EmoTaG, a few-shot emotion-aware 3D talking head synthesis framework built on the Pretrain-and-Adapt paradigm. Our key insight is to reformulate motion prediction in a structured FLAME parameter space rather than directly deforming 3D Gaussians, thereby introducing explicit geometric priors that improve motion stability. Building upon this, we propose a Gated Residual Motion Network (GRMN), which captures emotional prosody from audio while supplementing head pose and upper-face cues absent from audio, enabling expressive and coherent motion generation. Extensive experiments demonstrate that EmoTaG achieves state-of-the-art performance in emotional expressiveness, lip synchronization, visual realism, and motion stability.

Haolan Xu, Keli Cheng, Lei Wang, Ning Bi, Xiaoming Liu• 2026

Related benchmarks

TaskDatasetResultRank
Self-reconstructionNeutral Set talking videos 5s (train)
PSNR30.02
7
Self-reconstructionEmotional Set talking videos 5s (train)
PSNR29.95
7
3D talking head generationEmotional self-reconstruction (test)
Emotional Expressiveness4.5
5
Lip synchronizationOOD audio-driven Cross Identity
Sync-E Score9.133
5
Lip synchronizationOOD audio-driven Cross Language
Sync-E Score9.662
5
Emotional talking head reconstructionEmotion-intensity dataset (Level-1 Weaker Intensity)
PSNR30.01
4
Emotional talking head reconstructionEmotion-intensity Level-3 Stronger Intensity
PSNR29.92
4
Showing 7 of 7 rows

Other info

Follow for update