PutnamBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Formal Theorem Proving | PutnamBench | Solved Count668 | 56 | |
| Theorem Proving | PutnamBench Lean | Solved Rate668 | 23 | |
| Theorem Proving | PutnamBench (test) | Accuracy72 | 13 | |
| Theorem Proving | PutnamBench Number Theory | Solved Problems19 | 13 | |
| Theorem Proving | PutnamBench Lean 4 (full) | Solved Count668 | 12 | |
| Formal Theorem Proving | PutnamBench September 2025 | Solved Problems Count462 | 11 | |
| Theorem Proving | PutnamBench | Average Proof Length62.5 | 9 | |
| Mathematical formalization | PutnamBench 672 problems | C@163 | 8 | |
| Formal Mathematical Answer-Construction | PutnamBench | Solved Instances17 | 7 | |
| Autoformalization | PutnamBench (PB) | Mean Cycle Consistency0.561 | 6 | |
| Mathematical Theorem Proving | PutnamBench | Total Spend163 | 3 | |
| Automated Theorem Proving | PutnamBench Easy Mode | Solved Problems (Pass@32)43 | 3 | |
| Formal proof refactoring | PutnamBench 96 problems | Average Score4.19 | 2 | |
| Automated Theorem Proving | PutnamBench Hard Mode | Total Solved (Pass@32)36 | 2 | |
| Theorem Proving | PutnamBench 32-problem Lean 4 (random sample) | Solved Count32 | 1 |
Showing 15 of 15 rows