Finite Automata Extraction: Low-data World Model Learning as Programs from Gameplay Video
About
World models are defined as a compressed spatial and temporal learned representation of an environment. The learned representation is typically a neural network, making transfer of the learned environment dynamics and explainability a challenge. In this paper, we propose an approach, Finite Automata Extraction (FAE), that learns a neuro-symbolic world model from gameplay video represented as programs in a novel domain-specific language (DSL): Retro Coder. Compared to prior world model approaches, FAE learns a more precise model of the environment and more general code than prior DSL-based approaches.
Dave Goel, Matthew Guzdial, Anurag Sarkar• 2025
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Next-frame prediction | Pac-Man (train) | FID0.23 | 3 | |
| Next-frame prediction | Pac-Man (test) | FID0.28 | 3 | |
| Next-frame prediction | River Raid (train) | FID0.29 | 3 | |
| Next-frame prediction | River Raid (test) | FID0.25 | 3 | |
| World Modeling | River Raid (train) | FID0.14 | 3 | |
| World Modeling | River Raid (test) | FID0.13 | 3 | |
| World Modeling | Pac-Man (train) | FID0.28 | 3 | |
| World Modeling | Pac-Man (test) | FID0.29 | 3 |
Showing 8 of 8 rows