SOTA Tokenizer compression benchmarks and papers with code | WizwandBenchmarks
| Dataset Name | SOTA Method | Metric | Trend | | |
|---|
| Reasoning | | Bits per Token3.51 | | 5 | 4d ago |
| C4 fr | | Bits per Token (B/T)3.34 | | 5 | 4d ago |
| C4 ko | | C/T Ratio1.72 | | 5 | 4d ago |
| C4 ja | | C/T1.6 | | 5 | 4d ago |
| C4 zh | | C/T Ratio1.54 | | 5 | 4d ago |
| C4 en | | Bits per Token4.64 | | 5 | 4d ago |
Showing 6 of 6 rows
