Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Finite Automata Extraction: Low-data World Model Learning as Programs from Gameplay Video

About

World models are defined as a compressed spatial and temporal learned representation of an environment. The learned representation is typically a neural network, making transfer of the learned environment dynamics and explainability a challenge. In this paper, we propose an approach, Finite Automata Extraction (FAE), that learns a neuro-symbolic world model from gameplay video represented as programs in a novel domain-specific language (DSL): Retro Coder. Compared to prior world model approaches, FAE learns a more precise model of the environment and more general code than prior DSL-based approaches.

Dave Goel, Matthew Guzdial, Anurag Sarkar• 2025

Related benchmarks

TaskDatasetResultRank
Next-frame predictionPac-Man (train)
FID0.23
3
Next-frame predictionPac-Man (test)
FID0.28
3
Next-frame predictionRiver Raid (train)
FID0.29
3
Next-frame predictionRiver Raid (test)
FID0.25
3
World ModelingRiver Raid (train)
FID0.14
3
World ModelingRiver Raid (test)
FID0.13
3
World ModelingPac-Man (train)
FID0.28
3
World ModelingPac-Man (test)
FID0.29
3
Showing 8 of 8 rows

Other info

Follow for update