Pre-training on C4 (val)
17.8Perplexity
ADAMW + GradientStabilizer
Updated 6d ago
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
2025.02 | 17.8 | - | - | |
2025.02 | 17.83 | - | - | |
2025.02 | 18.62 | - | - | |
2025.02 | 18.76 | - | - | |
2025.02 | 18.84 | - | - | |
2025.02 | 18.84 | - | - | |
2025.02 | 18.89 | - | - | |
2025.02 | 18.89 | - | - | |
2025.02 | 18.9 | - | - | |
2025.02 | 18.98 | - | - | |
2025.02 | 19.01 | - | - | |
2025.02 | 19.04 | - | - | |
2025.02 | 19.15 | - | - | |
2025.02 | 19.21 | - | - | |
2025.02 | 19.23 | - | - | |
2025.02 | 19.24 | - | - | |
2025.02 | 19.42 | - | - | |
2025.02 | 20.13 | - | - | |
2025.02 | 20.48 | - | - | |
2025.02 | 21.35 | - | - | |
2025.02 | 21.39 | - | - | |
2025.02 | 21.57 | - | - | |
2025.02 | 21.84 | - | - | |
2025.02 | 22.29 | - | - | |
2025.02 | 23.14 | - | - | |
2025.02 | 23.32 | - | - | |
2025.02 | 23.86 | - | - | |
2025.02 | 23.89 | - | - | |
2025.02 | 24.09 | - | - | |
2025.02 | 24.16 | - | - | |
2025.02 | 24.17 | - | - | |
2025.02 | 24.31 | - | - | |
2025.02 | 24.32 | - | - | |
2025.02 | 24.34 | - | - | |
2025.02 | 24.48 | - | - | |
2025.02 | 24.6 | - | - | |
2024.05 | 25.08 | - | - | |
2024.05 | 25.29 | - | - | |
2024.05 | 25.36 | - | - | |
2024.05 | 25.88 | - | - | |
2025.02 | 26.66 | - | - | |
2025.02 | 26.82 | - | - | |
2025.02 | 28.04 | - | - | |
2025.02 | 28.05 | - | - | |
2025.02 | 28.26 | - | - | |
2025.02 | 28.31 | - | - | |
2025.02 | 28.34 | - | - | |
2025.02 | 28.52 | - | - | |
2025.02 | 28.72 | - | - | |
2025.02 | 28.72 | - | - | |
2025.02 | 28.95 | - | - | |
2025.02 | 29.02 | - | - | |
2024.05 | 33.52 | - | - | |
2024.05 | 33.76 | - | - | |
2024.05 | 33.92 | - | - | |
2024.05 | 34.35 | - | - | |
2024.05 | 34.88 | - | - | |
2024.05 | 34.99 | - | - | |
2026.04 | - | 41 | 3.21 | |
2026.04 | - | 40.73 | 3.55 | |
2026.04 | - | 40.73 | 3.23 | |
2026.04 | - | 37.1 | 3.28 | |
2026.04 | - | 89.31 | 3.03 | |
2026.04 | - | 88.22 | 3.39 | |
2026.04 | - | 88.22 | 2.86 | |
2026.04 | - | 79.46 | 3.02 |