INCLUDE
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multilingual Language Proficiency | INCLUDE base-44 | Average Score64.8 | 46 | |
| Factual knowledge | Include Lite | Seen Accuracy41.88 | 21 | |
| Language Understanding | INCLUDE base 44 | Average Score64.8 | 21 | |
| Language Understanding | Include_c | Accuracy47.26 | 12 | |
| Multiple Choice Question Answering | Include c | Normalized Accuracy30.43 | 10 | |
| Knowledge Evaluation | Include_c | Accuracy37.8 | 7 | |
| Natural Language Understanding | Include_c Spanish | Normalized Accuracy40.36 | 7 | |
| Multilingual Multiple-Choice Reasoning | INCLUDE 44 languages 1.0 (test) | Average Accuracy56.9 | 6 | |
| Isolated Sign Language Recognition | INCLUDE | Accuracy93.5 | 5 | |
| Multilingual Knowledge | INCLUDE | Accuracy77.2 | 4 | |
| Language Understanding | INCLUDE uk | Accuracy35.09 | 3 | |
| Language Understanding | INCLUDE (te) | Accuracy24.09 | 3 | |
| Language Understanding | INCLUDE (es) | Accuracy28 | 3 | |
| Language Understanding | INCLUDE (ru) | Accuracy26.99 | 3 | |
| Language Understanding | INCLUDE (hi) | Accuracy (INCLUDE hi)25.05 | 3 | |
| Knowledge Reasoning | Include c | Normalized Accuracy35.41 | 3 | |
| Natural Language Understanding | Include Spanish (test) | Accuracy38.91 | 3 | |
| Multilingual Language Understanding | INCLUDE 5-shot | Accuracy77.81 | 3 |
Showing 18 of 18 rows