MLVU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Understanding | MLVU | Score78.19 | 221 | |
| Long Video Understanding | MLVU | Accuracy83.8 | 205 | |
| Video Question-Answering | MLVU | Accuracy76.2 | 194 | |
| Video Understanding | MLVU | Accuracy81.5 | 114 | |
| Video Understanding | MLVU | Accuracy87.34 | 80 | |
| Long Video Understanding | MLVU (dev) | Score78.1 | 63 | |
| Long Video Understanding | MLVU (test) | Average Score81 | 60 | |
| Multi-discipline Long Video Understanding | MLVU | Score68.9 | 55 | |
| Video Understanding | MLVU 3-120min (test) | Accuracy47.7 | 49 | |
| Video Understanding | MLVU 3-120min (dev) | Accuracy63 | 49 | |
| Video Question Answering | MLVU 78 (test) | Accuracy76.66 | 45 | |
| Video Question Answering | MLVU | M-Avg Score72.4 | 40 | |
| Long-video Question Answering | MLVU | M-Avg79.5 | 39 | |
| Long Video Understanding | MLVU 3-120 min | Accuracy82.1 | 36 | |
| Video Question Answering | MLVU (dev) | Accuracy78.1 | 34 | |
| Video Understanding | MLVU (test) | Average100.3 | 34 | |
| Long Video Understanding | MLVU v1.0 (test) | MLVU Score67.77 | 28 | |
| Offline Video Understanding | MLVU v1 (test) | Accuracy71.5 | 26 | |
| Video Question Answering | MLVU 1.0 (test) | Accuracy68.5 | 26 | |
| Video Understanding | MLVU | Score78.1 | 24 | |
| Long Video Understanding | MLVU MCQ (test) | Accuracy81.5 | 22 | |
| Long Video Understanding | MLVU multiple-choice task | Overall Accuracy73.4 | 21 | |
| Video Understanding | MLVU MCQ (test) | Accuracy81.5 | 21 | |
| Video Understanding Reasoning | MLVU | Accuracy73.46 | 21 | |
| Video Understanding | MLVU (dev) | MLVU Dev Score68.4 | 21 |
Showing 25 of 56 rows