Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning

About

Generating natural and linguistically accurate sign language avatars remains a formidable challenge. Current Sign Language Production (SLP) frameworks face a stark trade-off: direct text-to-pose models suffer from regression-to-the-mean effects, while dictionary-retrieval methods produce robotic, disjointed transitions. To resolve this, we propose a novel training paradigm that leverages sparse keyframes to capture the true underlying kinematic distribution of human signing. By predicting dense motion from these discrete anchors, our approach mitigates regression-to-the-mean while ensuring fluid articulation. To realize this paradigm at scale, we first introduce FAST, an ultra-efficient sign segmentation model that automatically mines precise temporal boundaries. We then present SignSparK, a large-scale Conditional Flow Matching (CFM) framework that utilizes these extracted anchors to synthesize 3D signing sequences in SMPL-X and MANO spaces. This keyframe-driven formulation also uniquely unlocks Keyframe-to-Pose (KF2P) generation, making precise spatiotemporal editing of signing sequences possible. Furthermore, our adopted reconstruction-based CFM objective also enables high-fidelity synthesis in fewer than ten sampling steps; this allows SignSparK to scale across four distinct sign languages, establishing the largest multilingual SLP framework to date. Finally, by integrating 3D Gaussian Splatting for photorealistic rendering, we demonstrate through extensive evaluation that SignSparK establishes a new state-of-the-art across diverse SLP tasks and multilingual benchmarks.

Jianhe Low, Alexandre Symeonidis-Herzig, Maksym Ivashechkin, Ozge Mercanoglu Sincan, Richard Bowden• 2026

Related benchmarks

TaskDatasetResultRank
Text-to-PoseCSLDaily
DTW-PA-JPE (Body)4.87
8
Text-to-PosePHOENIX-2014T
DTW PA-JPE (Body)3.68
8
Text-to-PoseHow2Sign
DTW PA-JPE (Body)7.26
7
Sign StitchingPHOENIX-2014T (test)
DTW-J9.01
7
Sign Language SegmentationMeineDGS
mIoU77.2
5
Sign StitchingMeineDGS (unseen)
DTW-J8.57
5
Sign StitchingBSLCorpus (unseen)
DTW-J Distance5.24
5
Showing 7 of 7 rows

Other info

Follow for update