CMMLU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Chinese Multitask Language Understanding | CMMLU | Accuracy81.8 | 50 | |
| Language Understanding | CMMLU | Accuracy90.1 | 42 | |
| Multitask Language Understanding | CMMLU (test) | Accuracy78.3 | 38 | |
| General Knowledge | CMMLU | Accuracy89.5 | 25 | |
| Multi-task Language Understanding | CMMLU | Accuracy89.28 | 22 | |
| Examination | CMMLU | Score61.3 | 20 | |
| Knowledge | CMMLU | Knowledge Score84.72 | 16 | |
| Chinese Language Knowledge and Reasoning | CMMLU | Score77.01 | 14 | |
| General Language Understanding | CMMLU | Overall Accuracy77.3 | 14 | |
| Comprehensive Examination | CMMLU (test) | Accuracy68.1 | 14 | |
| Chinese Language Understanding | CMMLU (test) | CMMLU Score0.574 | 13 | |
| Chinese Language Understanding | CMMLU | Score90.9 | 10 | |
| General Reasoning | CMMLU (test) | Accuracy64.1 | 8 | |
| Comprehensive cognitive reasoning | CMMLU | Score53.45 | 8 | |
| Chinese multiple-choice evaluation | CMMLU | CMMLU College Mathematics Accuracy61.9 | 6 | |
| Question Answering | CMMLU | Accuracy88.1 | 6 | |
| Medical Knowledge Evaluation | CMMLU Med | Accuracy86.89 | 5 | |
| Chinese General Knowledge | CMMLU | Accuracy90.9 | 4 | |
| Knowledge & Reasoning | CMMLU | Accuracy63.4 | 4 | |
| General Domains | CMMLU | Accuracy0.865 | 4 | |
| General Language Understanding | CMMLU 5-shot | Accuracy90.61 | 3 | |
| Language Understanding | CMMLU Cantonese | Accuracy (Humanities)27.72 | 3 | |
| Downstream Performance Prediction | CMMLU | MSE0.0033 | 3 | |
| General Competence Evaluation | CMMLU | Accuracy84.94 | 2 | |
| Multilingual Understanding | CMMLU | Score72 | 2 |
Showing 25 of 27 rows