MMMLU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| General Knowledge Evaluation | MMMLU | MMMLU General Knowledge Accuracy82.25 | 29 | |
| Multilingual Language Understanding | MMMLU (Massive Multilingual Language Understanding) | Accuracy79.5 | 21 | |
| Multilingual Language Understanding | MMMLU | Accuracy (Korean)60.5 | 20 | |
| Multilingual Knowledge | MMMLU | Accuracy87.2 | 18 | |
| Multitask Language Understanding | MMMLU Swahili 1.0 (test) | Accuracy33.38 | 18 | |
| Multitask Language Understanding | MMMLU Korean 1.0 (test) | Accuracy41.94 | 18 | |
| Multitask Language Understanding | MMMLU non-EU languages (test) | Accuracy77.4 | 16 | |
| Multitask Language Understanding | MMMLU 24 official EU languages | Overall Score80.6 | 14 | |
| Chinese Language Understanding | MMMLU | MMMLU Score37.08 | 8 | |
| Question Answering | MMMLU | Accuracy36.14 | 8 | |
| Multilinguality | MMMLU ko, de, es, ja | Average Score86.3 | 4 | |
| Multilingual Language Understanding | MMMLU 5-shot | Accuracy78.94 | 3 |
Showing 12 of 12 rows