Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Phantom: Training Robots Without Robots Using Only Human Videos

About

Training general-purpose robots requires learning from large and diverse data sources. Current approaches rely heavily on teleoperated demonstrations which are difficult to scale. We present a scalable framework for training manipulation policies directly from human video demonstrations, requiring no robot data. Our method converts human demonstrations into robot-compatible observation-action pairs using hand pose estimation and visual data editing. We inpaint the human arm and overlay a rendered robot to align the visual domains. This enables zero-shot deployment on real hardware without any fine-tuning. We demonstrate strong success rates-up to 92%-on a range of tasks including deformable object manipulation, multi-object sweeping, and insertion. Our approach generalizes to novel environments and supports closed-loop execution. By demonstrating that effective policies can be trained using only human videos, our method broadens the path to scalable robot learning.

Marion Lepert, Jiaying Fang, Jeannette Bohg• 2025

Related benchmarks

TaskDatasetResultRank
Visual Fidelity EvaluationTACO and Aria Dataset
FD470.6
15
Robot ManipulationRoboTwin novel egocentric viewpoint 2.0
Adjust Success Rate52
6
Robot ManipulationRoboTwin standard egocentric viewpoint 2.0
Adjust96
6
Video GenerationRoboTwin Simulation novel egocentric viewpoint 2.0
PSNR16.97
5
DrawerAria (Real Robot) (test)
Success Rate5
4
FlowerAria (Real Robot) (test)
Success Rate (SR)0.00e+0
4
HammerAria (Real Robot) (test)
Success Rate (SR)0.00e+0
4
MustardAria (Real Robot) (test)
SR0.00e+0
4
Cross-embodiment video editingCross-embodiment video editing dataset
FVD1.95e+3
3
Showing 9 of 9 rows

Other info

Follow for update