F2F
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Automated Theorem Proving | MiniF2F (test) | Success Rate99.6 | 93 | |
| Theorem Proving | MiniF2F (val) | Success Rate63.9 | 59 | |
| Formal Theorem Proving | miniF2F Isabelle (val) | Success Rate57 | 41 | |
| Formal Theorem Proving | miniF2F Isabelle (test) | Success Rate51.2 | 39 | |
| Theorem Proving | miniF2F Lean (test) | Pass@6452 | 24 | |
| Autoformalization | miniF2F (test) | TC@196 | 16 | |
| Formal Theorem Proving | miniF2F (val) | Pass@142.2 | 15 | |
| Auto-formalization | MiniF2F (test) | Pass@8100 | 13 | |
| Informal-to-formal proving | miniF2F (val) | Proven Theorems Rate25.8 | 11 | |
| Theorem Proving | miniF2F Lean (val) | Cumulative Pass Rate60.2 | 10 | |
| Lean theorem proving | MINIF2F 244 problems | Pass@884.02 | 9 | |
| Informal-to-Formal Proving | miniF2F (test) | Accuracy24.6 | 6 | |
| Theorem Proving | miniF2F Lean (curriculum) | Pass@6432.1 | 3 |
Showing 13 of 13 rows