WISE: A Long-Horizon Agent in Minecraft with Why-Which Reasoning
About
Rapid advances have been made in developing general-purpose embodied agent in environments like Minecraft through the adoption of LLM-augmented hierarchical approaches. Despite their promise, low-level controllers often become performance bottlenecks due to repeated execution failures. We argue that a key limitation is not only the lack of episodic memory, but also the decoupling of \textit{what-where-when} memory from \textit{which-why} reasoning. To address this, we propose \textbf{WISE} (Which-Why Informed Semantic Explorer), a long-horizon agent framework with an enhanced low-level controller equipped with a Causal Event Graph that augments episodic memory with explicit causal structure linking observations to task relevance. Unlike prior work such as MrSteve, which relies on feature similarity for retrieval, WISE enables robust recall under viewpoint changes and supports opportunistic task reordering through causal reasoning. Building on this memory, we propose an Opportunistic Task Scheduler that dynamically re-prioritizes subtasks when causally relevant opportunities are detected. We further equip WISE with a multi-scale progressive exploration strategy to provide spatially comprehensive observations for downstream reasoning. Experiments show that WISE largely improves task success and efficiency on long-horizon sparse tasks, particularly in settings requiring adaptive decision-making.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Exploration | Minecraft 128 x 128 | Map Coverage97 | 8 | |
| Exploration | Minecraft 384 x 384 | Map Coverage83 | 8 | |
| Sparse Task Execution | ABA-Sparse (test) | Success Rate62 | 4 | |
| Sparse Task Execution | ABC-Sparse Adaptive Non-Sequential Task (test) | Success Rate77 | 4 |