MMVU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Understanding | MMVU | Accuracy75.4 | 76 | |
| Video reasoning | MMVU | Accuracy75.8 | 57 | |
| Video Question Answering | MMVU | Accuracy58.5 | 51 | |
| Video Reasoning | MMVU Multiple-choice (test) | Accuracy75.4 | 25 | |
| Video Question Answering | MMVU (val) | Average Score68.3 | 19 | |
| Multimodal Understanding | MMVU | Accuracy75.4 | 18 | |
| Video Reasoning | MMVU mc | Score82.6 | 16 | |
| Video Multimodal Understanding | MMVU | Accuracy68.6 | 15 | |
| Video Reasoning | MMVU (test) | Accuracy68.5 | 9 | |
| Video Question Answering | MMVU mc | Accuracy75.4 | 9 | |
| Video Understanding | MMVU (val) | Score46 | 9 | |
| Multi-discipline Video Understanding | MMVU | Accuracy75.4 | 7 | |
| Video Question Answering | MMVU (test) | Accuracy75.96 | 7 | |
| Expert-level Understanding | MMVU MC | Accuracy75.4 | 7 | |
| Multi-modal Video Understanding | MMVU | Score67 | 6 | |
| Video Question Answering | MMVU | M-Avg67.2 | 5 | |
| Video Reasoning | MMVU | Throughput (it/s)2.14 | 3 |
Showing 17 of 17 rows