A Task-State Representation for Long-Horizon Mobile GUI Agents
About
While long-horizon mobile GUI agents typically rely on thought-action-observation loops, they struggle to separate persistent task states from transient screen observations. As execution histories grow, this entanglement imposes a severe context burden, causing agents to forget initial requirements, hallucinate progress, or repeatedly interact with stale interfaces. To address this, we introduce Task-State Representation (TSR), a training-free framework that explicitly decouples task state from sensory input. Acting as a lightweight external wrapper, TSR maintains three structured components: a global instruction summary, a dynamic progress tracker for subgoals, and a transition-aware action verifier. By continuously updating through pre- and post-action visual comparisons, TSR effectively guides the agent's reasoning without requiring architectural modifications. Experiments across four mobile GUI benchmarks validate TSR's effectiveness, yielding up to a 12 absolute point increase in success rate on complex cross-application and memory-intensive tasks.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| End-to-end GUI Navigation | AndroidWorld | Success Rate57.76 | 25 | |
| Mobile GUI Agent Navigation | MobileWorld | Success Rate (SR)58 | 4 | |
| Mobile GUI Agent Navigation | MemGUI-Bench | Success Rate42.97 | 4 | |
| Mobile GUI Agent Navigation | MemGUI Memory | Success Rate (SR)40.87 | 4 | |
| Mobile GUI Agent Navigation | VenusBench Mobile | Success Rate (SR)26.27 | 4 |