SlimPajama
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Language Modeling | SlimPajama | Perplexity (PPL)2.97 | 77 | |
| Language Modeling | SlimPajama latest (val) | Validation Loss3.078 | 26 | |
| Language Modeling | SlimPajama (test) | PPL (CommonCrawl)0.5402 | 23 | |
| Language Modeling | SlimPajama (val) | Perplexity3.09 | 13 | |
| Language Modeling | SlimPajama large-scale (train) | L(ψ)2.129 | 8 | |
| Language Modeling | SlimPajama-672B (val) | Validation Perplexity8.09 | 6 | |
| Factuality Evaluation | SlimPajama | Pointwise Score63.5 | 3 | |
| Generation Quality and Coherence Evaluation | SlimPajama Quality Evaluation (test) | Gen Quality (Std. Prefix)86.3 | 3 | |
| Language Modeling | SlimPajama 10M (dev) | Perplexity9.219 | 3 | |
| Self-attention inverse temperature scaling analysis | SlimPajama | Tie Percentage6 | 2 | |
| Language Modeling | SlimPajama 6B (val) | Validation Perplexity (1B tokens)38.47 | 2 | |
| Language Modeling | SlimPajama-6B (train) | Train Loss (1B tokens)3.778 | 2 |
Showing 12 of 12 rows