Abstraction Generation for Generalized Planning with Pretrained Large Language Models
About
Qualitative Numerical Planning (QNP) serves as an important abstraction model for generalized planning (GP), which aims to compute general plans that solve multiple instances at once. Recent works show that large language models (LLMs) can function as generalized planners. This work investigates whether LLMs can serve as QNP abstraction generators for GP problems and how to fix abstractions via automated debugging. We propose a prompt protocol: input a GP domain and training tasks to LLMs, prompting them to generate abstract features and further abstract the initial state, action set, and goal into QNP problems. An automated debugging method is designed to detect abstraction errors, guiding LLMs to fix abstractions. Experiments demonstrate that under properly guided by automated debugging, some LLMs can generate useful QNP abstractions.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| QNP abstraction generation | Delivery domain (evaluation) | Coverage Rate100 | 8 | |
| QNP abstraction generation | Heavy domain (evaluation) | Coverage Rate100 | 8 | |
| QNP abstraction generation | Gripper domain (evaluation) | Coverage Rate85 | 8 | |
| QNP abstraction generation | Miconic domain (eval) | Coverage Rate40 | 8 | |
| QNP abstraction generation | Ferry domain (evaluation) | Coverage Rate1 | 8 | |
| QNP abstraction generation | Spanner domain (test) | Coverage Rate0.00e+0 | 8 | |
| QNP abstraction generation | Forest domain (val) | Coverage Rate0.00e+0 | 8 |