Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

HiF-VLA: Hindsight, Insight and Foresight through Motion Representation for Vision-Language-Action Models

About

Vision-Language-Action (VLA) models have recently enabled robotic manipulation by grounding visual and linguistic cues into actions. However, most VLAs assume the Markov property, relying only on the current observation and thus suffering from temporal myopia that degrades long-horizon coherence. In this work, we view motion as a more compact and informative representation of temporal context and world dynamics, capturing inter-state changes while filtering static pixel-level noise. From this perspective, HiF-VLA equips a motion-centric world model for the VLA, enabling agents to reason about temporal dynamics for future evolution during action generation. Building on this idea, we propose HiF-VLA (Hindsight, Insight, and Foresight for VLAs), a unified framework that leverages motion for bidirectional temporal reasoning. HiF-VLA encodes past dynamics through hindsight priors, anticipates future motion via foresight reasoning, and integrates both through a hindsight-modulated joint expert to enable a ''think-while-acting'' paradigm for long-horizon manipulation. As a result, HiF-VLA surpasses strong baselines on LIBERO-Long and CALVIN ABC-D benchmarks, while incurring negligible additional inference latency. Furthermore, HiF-VLA achieves substantial improvements in real-world long-horizon manipulation tasks, demonstrating its broad effectiveness in practical robotic settings.

Minghui Lin, Pengxiang Ding, Shu Wang, Zifeng Zhuang, Yang Liu, Xinyang Tong, Wenxuan Song, Shangke Lyu, Siteng Huang, Donglin Wang• 2025

Related benchmarks

TaskDatasetResultRank
Robotic ManipulationLIBERO v1 (test)
Average Success Rate98
46
Sequential Robotic ManipulationCALVIN
Success Rate (1 task)93.5
45
Robotic ManipulationLIBERO Long--
44
Instruction-following robotic manipulationCALVIN ABC→D (unseen environment D)
Success Rate (Length 1)98.5
29
Robot ManipulationLIBERO LONG (test)
Success Rate96.4
15
Robot ManipulationMetaWorld v1 (train test)
Success Rate (Easy)72.9
8
Showing 6 of 6 rows

Other info

GitHub

Follow for update