Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation
About
In robotic manipulation, the tight coupling between grasping and motion planning often obscures the true source of failure, leading to inefficient trial-and-error. To enable efficient long-horizon manipulation, we propose GTP-FA (Grasp-Then-Plan with Failure Attribution), a task-oriented two-stage grasp-then-plan framework that generates grasp candidates and performs downstream motion planning conditioned on the selected grasp. Given a failed manipulation trajectory, we learn a failure attribution model that generalizes to unseen grasps and produces a stable distribution over failure modes for diagnosis-guided optimization. Based on these attribution results, we then optimize both modules in a diagnosis-driven manner: on the grasping side, we inject task-level priors and risk penalties into grasp candidate scoring and optimization to suppress unstable or task-incompatible grasps; on the planning side, we target high-risk initial states through data collection and fine-tuning to address genuine planning bottlenecks. We evaluate the proposed framework in both simulation and real-robot experiments, and show that GTP-FA improves the corresponding base learners across RL, IL, diffusion-policy, and VLA-based settings, achieving substantially higher overall task success rates.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Stack | ManiSkill 3 | Success at End95.2 | 25 | |
| Poke | ManiSkill3 | Success Rate (At End)76.6 | 10 | |
| LiftPeg | ManiSkill3 | Success Rate (At End)71.2 | 5 | |
| Average performance across 5 manipulation tasks | Real-robot Average | Success Rate76.8 | 2 | |
| Pick | ManiSkill3 | Success Rate (End)96.6 | 2 | |
| Pick up the red end of the stick and push the yellow cube into the red target area | Real-robot POKECUBE | Success Rate86 | 2 | |
| Pick up the red handle of the gray cup and pour the contents into the blue-gray cup | Real-robot POURWATER | Success Rate54 | 2 | |
| Pick up the red part of the hook and pull the yellow cube into the red target area | Real-robot PULLCUBETOOL | Success Rate78 | 2 | |
| Place | ManiSkill 3 | Success At End92.1 | 2 | |
| Place the orange into the pink tray | Real-robot Orange-to-tray | Success Rate92 | 2 |