WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents
About
To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing long-term memory benchmarks mainly evaluate language-centric retrieval and question answering, while embodied benchmarks often focus on short-horizon task execution without testing long-term memory use in dynamic environments. We introduce WorldLines, a project-driven benchmark for long-horizon embodied household assistance. It constructs temporally extended household traces with dialogues, actions, execution feedback, object and device state changes, and converts them into evidence-linked samples for Memory QA and Embodied Task Planning. We further propose ObsMem, an observer-grounded memory framework that maintains visibility-aware memories and action-native state trails for state-aware decisions. Experiments reveal persistent challenges in partial observability, overwritten world states, and translating long-term memory into embodied plans, while ObsMem offers a stronger reference architecture for this setting.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Embodied Planning | WorldLines 21 action-dense samples downstream planning probe | Plan Judge68.4 | 5 | |
| Embodied Task Planning | WorldLines Downstream Embodied (planning set) | Plan Judge Score68.4 | 5 | |
| Memory Question Answering | WorldLines Memory QA samples | Judge Score71.3 | 5 |