| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| C4 Llama 2 pre-training (val) | RMSProp+Magma | Perplexity13.19 | 47 | 1mo ago | |
| C4 Llama-160M scratch (val) | GPA-AdamW | Validation Loss3.0908 | 20 | 1mo ago | |
| C4 Llama-1B scratch (val) | SF-AdamW | Validation Loss2.638 | 16 | 1mo ago | |
| BERT-Large NVIDIA V100 (train) | Checkpoint | Max Batch Size96 | 6 | 1mo ago | |
| BERT-Large NVIDIA 2080 Ti (train) | Checkpoint | Max Batch Size50 | 6 | 1mo ago | |
| Datacomp-LM 400M-1x scale 1.0 | DCLM-baseline + SIEVE | Core Score17.9 | 3 | 1mo ago |
