| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Language Modeling | Proof-pile | Perplexity2.33 | 58 | |
| Language Modeling | Proof-pile (test) | Perplexity (2K)2.91 | 16 | |
| Language Modeling | Proof-pile 128 32k-length documents (test) | Perplexity2.28 | 8 | |
| Language Modeling | Proof-Pile | PPL2.915 | 6 |