Pre-training
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Language Modeling | Pre-training (val) | Validation Loss1.602 | 55 | |
| Pre-training | Pre-training (evaluation) | Pre-training Eval Loss3.254 | 5 | |
| Pre-training efficiency | Pre-training | Muon Steps4,228 | 4 |
Showing 3 of 3 rows