Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Mathematical Reasoning on Math lm-evaluation-harness (GSM8k, MBPP, MATH)
Loading...
1.44
GSM8k Accuracy
Transformer
1.2008
1.2629
1.325
1.3871
Jan 25, 2026
GSM8k Accuracy
MBPP Accuracy
MATH Accuracy
Updated 1mo ago
Evaluation Results
Method
Method
Links
GSM8k Accuracy
MBPP Accuracy
MATH Accuracy
Transformer
Total parameters=918m,...
2026.01
1.44
0
0.08
Transformer + Spelling Bee Embeddings
Total parameters=918m,...
2026.01
1.21
0.2
0.12
Feedback
Search any
task
Search any
task
