WikiText
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Language Modeling | WikiText-2 (test) | PPL1.2 | 2,416 | |
| Language Modeling | WikiText | PPL0.2838 | 740 | |
| Language Modeling | WikiText | Word Perplexity2.9 | 331 | |
| Language Modeling | Wikitext | Wikitext PPL7.21 | 151 | |
| Language Modeling | WikiText-2 | Perplexity4.88 | 105 | |
| Language Modeling | WikiText (test) | Perplexity4.88 | 69 | |
| Language Modeling | WikiText (val) | Perplexity12.51 | 62 | |
| Language Modeling | WikiText | Perplexity0.1 | 47 | |
| Language Modeling | WikiText v1 (test) | Perplexity13.33 | 30 | |
| Language Modeling | WikiText (held-out) | Perplexity (PPL)9.8 | 25 | |
| Language Modeling | Wikitext zero-shot | Perplexity24.01 | 22 | |
| Language Modeling | WikiText-103 | Throughput (tokens/s)159,000 | 21 | |
| Linear Mode Connectivity | WikiText | Loss Barrier0.18 | 20 | |
| Open-ended Text Generation | WikiText | Rep-254.83 | 18 | |
| Language Modeling | WikiText-103 Clean (val) | Perplexity31.82 | 18 | |
| Language Modeling | WikiText (WT) | Relative PPL Change (%)31 | 16 | |
| Language Modeling | WikiText | PPL Change (%)1.7 | 16 | |
| Language Modeling | WikiText-103 | Bits Per Character (BPC)2 | 13 | |
| Language Modeling | WikiText-2 vLLM harness (test) | Perplexity (PPL)8.87 | 12 | |
| Privacy Measurement | WikiText | Epsilon0 | 12 | |
| Open-ended Text Generation | Wikitext (test) | Diversity (DIV)95 | 12 | |
| Language Modeling | WikiText | PPL17.69 | 11 | |
| Language Modeling | WikiText-103 20w x 2048 | Perplexity (PPL)9.603 | 10 | |
| Prefilling Profiling | WikiText (test) | Time (s)38 | 10 | |
| Language Modeling | WikiText 1,000-example evaluation slice (test) | Perplexity12.723 | 9 |
Showing 25 of 57 rows