A Minimal Agent for Automated Theorem Proving
About
We propose a minimal agentic baseline that enables systematic comparison across different AI-based theorem prover architectures. This design implements the core features shared among state-of-the-art systems: iterative proof refinement, library search and context management. We evaluate this agentic approach using qualitatively different benchmarks and compare various frontier language models and design choices. Our results show competitive performance compared to state-of-the-art approaches, while using a significantly simpler architecture. Additionally, we demonstrate consistent advantages of an iterative approach over multiple single-shot generations, especially in terms of sample efficiency and cost effectiveness. The implementation is released open-source as a candidate reference for future research and as an accessible prover for the community.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Theorem Proving | PutnamBench Lean | Solved Rate91 | 23 | |
| Formal Theorem Proving | PutnamBench | Solve Rate54.7 | 14 | |
| Formal Theorem Proving | Fate-H | Solve Rate66 | 7 | |
| Automated Theorem Proving | FATE-M | Pass Rate98 | 5 | |
| Automated Theorem Proving | Fate-X | Pass Rate24 | 5 | |
| Automated Theorem Proving | LeanCAT | Pass Rate59 | 2 |