Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model

About

Text-driven human motion generation in computer vision is both significant and challenging. However, current methods are limited to producing either deterministic or imprecise motion sequences, failing to effectively control the temporal and spatial relationships required to conform to a given text description. In this work, we propose a fine-grained method for generating high-quality, conditional human motion sequences supporting precise text description. Our approach consists of two key components: 1) a linguistics-structure assisted module that constructs accurate and complete language feature to fully utilize text information; and 2) a context-aware progressive reasoning module that learns neighborhood and overall semantic linguistics features from shallow and deep graph neural networks to achieve a multi-step inference. Experiments show that our approach outperforms text-driven motion generation methods on HumanML3D and KIT test sets and generates better visually confirmed motion to the text conditions.

Yin Wang, Zhiying Leng, Frederick W. B. Li, Shun-Cheng Wu, Xiaohui Liang• 2023

Related benchmarks

TaskDatasetResultRank
Text-to-motion generationHumanML3D (test)
FID0.243
331
text-to-motion mappingKIT-ML (test)
R Precision (Top 3)0.745
275
text-to-motion mappingHumanML3D (test)
FID0.243
243
Text-to-motion generationKIT-ML (test)
FID0.571
115
Text-to-motion generationHumanML3D 19 (test)
FID0.243
37
Text-conditional motion synthesisHumanML3D 12 (test)
R-Precision Top-149.2
15
Text-conditional motion synthesisHumanML3D 16 (test)
R-Precision Top-10.492
15
Text-to-motion generationKIT-ML 52 (test)
R-Precision Top-10.418
11
Showing 8 of 8 rows

Other info

Follow for update