RACE
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Reading Comprehension | RACE high | Accuracy94.5 | 295 | |
| Reading Comprehension | RACE mid | Accuracy89.9 | 196 | |
| Reading Comprehension | RACE | Accuracy89.95 | 151 | |
| Machine Reading Comprehension | RACE (test) | RACE Accuracy (Medium)95.4 | 111 | |
| Multiple Choice Question Answering | RACE | Accuracy98.24 | 46 | |
| Machine Reading Comprehension | RACE | RACE Overall Accuracy94.5 | 38 | |
| Reading Comprehension | RACE | Accuracy74.93 | 34 | |
| Reading Comprehension | RACE-m | Accuracy0.931 | 28 | |
| Reading Comprehension | RACE-h (test) | Accuracy89.95 | 24 | |
| Understanding | RACE Middle | Score67.27 | 20 | |
| Question Answering | RACE-C | Accuracy93.66 | 19 | |
| Reading Comprehension | Race M | Race M Score45.47 | 18 | |
| Reading Comprehension | Race-H | RACE-h Score38.56 | 18 | |
| Reading Comprehension | RACE Middle School | Accuracy (RACE MS)95.4 | 16 | |
| Reading Comprehension | RACE (dev) | Accuracy88.1 | 16 | |
| Reading Comprehension | RACE-h | Accuracy62.3 | 15 | |
| Question Answering | RACE MRQA out-of-domain evaluation | EM46.3 | 15 | |
| Reading Comprehension | Race 3shots | Accuracy83.31 | 14 | |
| Reading Comprehension | RACE MRQA out-of-domain | EM43.62 | 14 | |
| Reading Comprehension | RACE | Vanilla Accuracy83.36 | 14 | |
| Machine Reading Comprehension | RACE Full (test) | Accuracy53.3 | 13 | |
| Reading Comprehension | RACE | RACE Middle Score70.2 | 12 | |
| Multiple Choice Question Answering | RACE++ | Accuracy68.3 | 11 | |
| Reading Comprehension | RACE | Exact Match83.08 | 11 | |
| Audio Question Answering | RACE Audio | Accuracy (Sub-task 1)64.21 | 10 |
Showing 25 of 45 rows