Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Pretraining on C4
Loading...
16.79
Perplexity
HTMuon
16.1732
20.3366
24.5
28.6634
Mar 10, 2026
Perplexity
Updated 1mo ago
Evaluation Results
Method
Method
Links
Perplexity
HTMuon
Model=LLaMA-350M
2026.03
16.79
Muon
Model=LLaMA-350M
2026.03
16.81
AdamW
Model=LLaMA-350M
2026.03
16.96
Adam
Model=LLaMA-350M
2026.03
17.11
HTMuon
Model=LLaMA-135M
2026.03
21.25
Muon
Model=LLaMA-135M
2026.03
22.23
Adam
Model=LLaMA-135M
2026.03
23.01
AdamW
Model=LLaMA-135M
2026.03
23.33
HTMuon
Model=LLaMA-60M
2026.03
27.88
Muon
Model=LLaMA-60M
2026.03
28.8
AdamW
Model=LLaMA-60M
2026.03
31.85
Adam
Model=LLaMA-60M
2026.03
32.21
Feedback
Search any
task
Search any
task
