Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Mathematical Reasoning on GSM8K (test) (PPL, pass@1, Throughput)
Loading...
1.25
Perplexity (PPL)
AR Transformer
1.216
1.4455
1.675
1.9045
Jul 2, 2026
Perplexity (PPL)
Pass Rate @1
Throughput
Updated 23d ago
Evaluation Results
Method
Method
Links
Perplexity (PPL)
Pass Rate @1
Throughput
AR Transformer
2026.07
1.25
75.74
67.16
BD3LM
S=4
2026.07
1.41
63.53
55.39
SW-SetDLM
S=≤ 8
2026.07
1.42
66.41
60.42
SW-SetDLM
S=≤ 16
2026.07
1.5
63.91
56.5
BD3LM
S=8
2026.07
1.53
56.94
54.67
SW-SetDLM
S=≤ 32
2026.07
1.59
61.94
55.08
BD3LM
S=16
2026.07
1.67
50.49
52.18
MDLM
decoding_strategy=conf...
2026.07
2.1
6.37
24.48
Feedback
Search any
task
Search any
task