Language Modeling Pre-training on C4 (val)
13.33PPL (60k)
FOAM-2
Updated 19d ago
Evaluation Results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2025.12 | 13.33 | - | 15.78 | 12.01 | 11.39 | 11.13 | 270.2 | - | - | - | - | - | - | - | |
2025.12 | 13.46 | - | 15.71 | 12.29 | 11.98 | - | 126.2 | - | - | - | - | - | - | - | |
2025.01 | 14.8 | 8.54 | - | - | 13.21 | - | 279.1 | 0.532 | - | - | 19.74 | 16.41 | 13.77 | 13.46 | |
2025.12 | 15.17 | - | 18.49 | 13.54 | 12.8 | 12.63 | 270.1 | - | - | - | - | - | - | - | |
2025.12 | 15.4 | - | 18.14 | 14.13 | 14.02 | - | 297.3 | - | - | - | - | - | - | - | |
2025.12 | 15.4 | - | 19.49 | 14.09 | 13.75 | - | 125.6 | - | - | - | - | - | - | - | |
2025.01 | 15.4 | 9.28 | - | - | 13.75 | - | 274.4 | 0.541 | - | - | 20.78 | 17.19 | 14.4 | 13.9 | |
2025.12 | 15.63 | - | 18.35 | 14.33 | 13.77 | 13.69 | 341.8 | - | - | - | - | - | - | - | |
2025.12 | 15.69 | - | 18.63 | 14.36 | 14.19 | - | 130.1 | - | - | - | - | - | - | - | |
2025.01 | 15.7 | 10.7 | - | - | 14.31 | - | 541.9 | 0.274 | - | - | 21.79 | 17.44 | 14.99 | 14.52 | |
2025.12 | 15.71 | - | 18.16 | 13.87 | 13.23 | 13.19 | 350 | - | - | - | - | - | - | - | |
2025.12 | 15.98 | - | 18.44 | 14.9 | 14.73 | - | 143.6 | - | - | - | - | - | - | - | |
2025.01 | 15.98 | 9.28 | - | - | 14.73 | - | 335.4 | 0.526 | - | - | 20.42 | 17.36 | 15.12 | 14.79 | |
2025.12 | 16.13 | - | 18.73 | 14.56 | 13.4 | 13.23 | 285.5 | - | - | - | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.968 | 19.459 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.923 | 18.596 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.932 | 18.765 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.934 | 18.803 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.929 | 18.698 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 3.003 | 20.145 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.93 | 18.727 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.976 | 19.609 | - | - | - | - | |
2025.06 | - | - | - | - | - | - | - | - | 2.918 | 18.504 | - | - | - | - | |
2025.01 | - | 16 | - | - | - | - | - | - | - | - | - | - | - | - |