Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Mathematical Problem Solving on Putnam-Axiom (PA)
Loading...
39.5
Exact Match Acc
DeepMath (RULES)
5.388
14.244
23.1
31.956
Feb 1, 2026
Exact Match Acc
Updated 1mo ago
Evaluation Results
Method
Method
Links
Exact Match Acc
DeepMath (RULES)
Optimization=RULES
2026.02
39.5
DeepMath (vanilla)
Optimization=vanilla
2026.02
39.1
DeepMath (GRPO)
Optimization=GRPO
2026.02
37.9
Qwen2.5 (GRPO)
Optimization=GRPO
2026.02
35.6
Qwen2.5 (vanilla)
Optimization=vanilla
2026.02
32.4
Qwen2.5 (RULES)
Optimization=RULES
2026.02
32
Llama-3.1 (GRPO)
Optimization=GRPO
2026.02
13.2
Llama-3.1 (RULES)
Optimization=RULES
2026.02
12.8
Llama-3.1 (vanilla)
Optimization=vanilla
2026.02
11.5
OLMO2 (vanilla)
Optimization=vanilla
2026.02
8.4
OLMO2 (GRPO)
Optimization=GRPO
2026.02
7.8
OLMO2 (RULES)
Optimization=RULES
2026.02
6.7
Feedback
Search any
task
Search any
task