Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

WANDR: Intention-guided Human Motion Generation

About

Synthesizing natural human motions that enable a 3D human avatar to walk and reach for arbitrary goals in 3D space remains an unsolved problem with many applications. Existing methods (data-driven or using reinforcement learning) are limited in terms of generalization and motion naturalness. A primary obstacle is the scarcity of training data that combines locomotion with goal reaching. To address this, we introduce WANDR, a data-driven model that takes an avatar's initial pose and a goal's 3D position and generates natural human motions that place the end effector (wrist) on the goal location. To solve this, we introduce novel intention features that drive rich goal-oriented movement. Intention guides the agent to the goal, and interactively adapts the generation to novel situations without needing to define sub-goals or the entire motion path. Crucially, intention allows training on datasets that have goal-oriented motions as well as those that do not. WANDR is a conditional Variational Auto-Encoder (c-VAE), which we train using the AMASS and CIRCLE datasets. We evaluate our method extensively and demonstrate its ability to generate natural and long-term motions that reach 3D goals and generalize to unseen goal locations. Our models and code are available for research purposes at wandr.is.tue.mpg.de.

Markos Diomataris, Nikos Athanasiou, Omid Taheri, Xi Wang, Otmar Hilliges, Michael J. Black• 2024

Related benchmarks

TaskDatasetResultRank
Motion GenerationGIMO curated P&R sequences
Prime Success19.04
8
Motion GenerationMoGaze curated P&R sequences
Prime Success11.26
8
Motion GenerationADT curated P&R sequences
Prime Success14.94
8
Motion GenerationHD-EPIC curated P&R sequences
Prime Success10.21
8
Motion GenerationHOT3D curated P&R sequences
Prime Success12.26
8
Goal-reaching motion generationGoal-reaching (evaluation set)
SR56.16
6
Long-term Sequential Motion GenerationCIRCLE Goal 2
SR37.33
2
Long-term Sequential Motion GenerationCIRCLE Goal 3
Success Rate (SR)20.96
2
Showing 8 of 8 rows

Other info

Code

Follow for update