Indic-English evaluation corpus
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Tokenization Efficiency | Indic-English evaluation corpus ta | NSL Score56 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus san | NSL score0.94 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus pa | NSL Score89 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus nep | NSL Score96 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus mr | NSL Score94 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus ml | NSL Score73 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus mai | NSL Score99 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus kn | NSL Score81 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus kas | NSL Score1.19 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus hi | NSL Score0.94 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus gu | NSL Score76 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus gom | NSL Score1.13 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus code | NSL Score2.09 | 5 | |
| Tokenization Efficiency | Indic-English evaluation corpus bn | NSL Score83 | 5 |
Showing 14 of 14 rows