| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| C4 (val) | ADAMW + GradientStabilizer | Perplexity17.8 | 58 | 3mo ago | |
| SOTA Model Pre-training Corpora | GShard | Representative Energy (MWh)24.1 | 5 | 1mo ago | |
| Pre-training (evaluation) | BHyT | Pre-training Eval Loss3.254 | 5 | 5mo ago | |
| C4 | GrassJump | Evaluation Loss (10k Steps)4.86 | 3 | 25d ago | |
| FineWeb 124M Transformer (val) | Training Time to Loss 3.28 (min)2.1 | 3 | 5mo ago |
