| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Long-context Input (Summarization) | PG19 | TPT6.12 | 20 | |
| Language Modeling | PG19 tokens (test) | Bits per Token3.49 | 14 | |
| Language Modeling | PG19 bytes (test) | Bits Per Token0.935 | 14 | |
| Language Modeling | PG19 T5 (val) | PPLX15.31 | 10 | |
| Lossless text compression | PG19 | Compression Ratio (bits)6.77 | 5 |