Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

RHO: Your Coding Agent is Secretly a Roboticist

About

Code-as-Policies (CaP) has shown that large language models (LLMs) can write code to solve robotics tasks by composing perception, planning, and control primitives. Recent CaP systems, however, rely on multi-turn code-generation loops at test time, which is often infeasible for real-time robot control. We introduce Robotics Harness Optimization (RHO), a novel paradigm in which tool-enabled coding agents, at training time, propose and search for interpretable, neurosymbolic multi-file policy repositories (Repositories-as-Policies) that compose these primitives rather than a single prompt, function, or file. RHO searches with reflective feedback from environment reward and execution rather than teleoperation demonstrations. It generalizes to perturbed pick-and-place settings like LIBERO-PRO, where OpenVLA scores 0.0% and $\pi_{0.5}$ averages 12.83%. Using the same low-level primitives, RHO reaches a 45.0% success rate, 2.5x higher than the strongest multi-turn agentic system, and 3.5x higher than $\pi_{0.5}$. On Robosuite, RHO sets a new state-of-the-art of 70.0%, exceeding the prior multi-turn record of 68.29% using single-turn execution with no corrective LLM code edits at deployment. When an LLM is used in the control loop, as on RAI's O3DE benchmark, RHO optimizes the deployed agent's multi-file harness of prompts, tools, and control code, improving held-out success from 23.5% to 44.3% with 20% less wall-clock time and 27% fewer tool calls.

Karim Elmaaroufi, Justin Svegliato, Sarunas Kalade, Graham Schelle, Sanjit A. Seshia, Matei Zaharia• 2026

Related benchmarks

TaskDatasetResultRank
Robot ManipulationLIBERO Object--
139
Robotic ManipulationLIBERO Goal
Positional Success Rate100
55
AverageLIBERO Spatial
Positional Success Rate53
5
Pick(between(plate,ramekin), plate)LIBERO Spatial
Pos Success Rate92
5
Pick(drawer_top(cabinet_wood), plate)LIBERO Spatial
Pos Success Rate6
5
Pick(next_to(plate), plate)LIBERO Spatial
Pos Success Rate56
5
Pick(next_to(ramekin), plate)LIBERO Spatial
Pos Success Rate78
5
Pick(on(stove), plate)LIBERO Spatial
Pos Success Rate86
5
Pick(table_center, plate)LIBERO Spatial
Pos Success Rate96
5
Pick(on(cabinet_wood), plate)LIBERO Spatial
Pos Success Rate42
5
Showing 10 of 14 rows

Other info

Follow for update