Languages
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Acoustic Discriminability (ABX) | 5 Languages (sw, ta, th, tr, uk) (dev) | Triphone ABX (WS)3.58 | 22 | |
| Within-Speaker ABX | 3 languages (test) | Avg ABX (w/o 0h)3.76 | 7 | |
| Natural Language Inference | 20 languages | Accuracy53.5 | 6 | |
| Named Entity Recognition | 20 languages | F1 Score66.3 | 6 | |
| Part-of-Speech Tagging | 6 languages Averaged (test) | F1 Score69.2 | 4 | |
| Sentence Retrieval | 6 languages Same script split | Accuracy43.9 | 4 | |
| Natural Language Inference | 6 languages All transfers | Accuracy43 | 4 | |
| Dependency Labeling | 6 languages All transfers | F1 Score18.1 | 4 | |
| Dependency Labeling | 6 languages (Different script) | F1 Score15.9 | 4 | |
| Part-of-Speech Tagging | 6 languages Same script split | F1 Score41.9 | 4 | |
| Cross-language Vocabulary Overlap | 6 languages All transfers | JSD0.74 | 4 | |
| Cross-language Vocabulary Overlap | 6 languages (Same script split) | JSD0.62 | 4 | |
| Cross-language Vocabulary Overlap | 6 languages Different script | JSD0.77 | 4 | |
| Sentence Retrieval | 20 languages All transfers | Accuracy45.1 | 3 | |
| Sentence Retrieval | 20 languages Different script | Accuracy44.1 | 3 | |
| Natural Language Inference | 20 languages Different script | Accuracy37.8 | 3 | |
| Dependency Labeling | 20 languages Same script | F1 Score19.4 | 3 | |
| Dependency Labeling | 20 languages Different script | F1 Score16.5 | 3 | |
| Named Entity Recognition | 20 languages Same script | F1 Score54.3 | 3 | |
| Vocabulary Overlap | 20 languages Same script | JSD0.58 | 3 | |
| Vocabulary Overlap | 20 languages Different script | JSD0.75 | 3 | |
| Dependency Labeling | 20 languages | F1 Score0.545 | 3 | |
| Part-of-Speech Tagging | 20 languages | F1 Score67.3 | 3 | |
| Masked Language Modeling | 20 languages | MRR52.6 | 3 | |
| Vocabulary Allocation | 20 languages | AR809 | 3 |
Showing 25 of 25 rows