MLVU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Long Video Understanding | MLVU | Accuracy83.8 | 72 | |
| Video Understanding | MLVU | Score50 | 70 | |
| Video Understanding | MLVU | M-AVG70.8 | 54 | |
| Video Question-Answering | MLVU | Accuracy74.5 | 53 | |
| Video Understanding | MLVU 3-120min (test) | Accuracy47.7 | 49 | |
| Video Understanding | MLVU 3-120min (dev) | Accuracy63 | 49 | |
| Video Question Answering | MLVU 78 (test) | Accuracy76.66 | 45 | |
| Long Video Understanding | MLVU (test) | Average Score55.8 | 41 | |
| Video Understanding | MLVU (test) | Average100.3 | 34 | |
| Long Video Understanding | MLVU (dev) | Score78.1 | 31 | |
| Long Video Understanding | MLVU 3-120 min | Accuracy82.1 | 23 | |
| Long-video Question Answering | MLVU | M-Avg77.3 | 22 | |
| Long Video Understanding | MLVU multiple-choice task | Overall Accuracy73.4 | 21 | |
| Video Understanding | MLVU MCQ (test) | Accuracy81.5 | 21 | |
| Video Understanding | MLVU (dev) | MLVU Dev Score68.4 | 17 | |
| Video Understanding | MLVU | MAT12.42 | 16 | |
| Video Question Answering | MLVU | M-Avg Score72.1 | 16 | |
| Video Question Answering | MLVU long-form | Accuracy74.7 | 14 | |
| Long Video Question Answering | MLVU multiple-choice questions | Accuracy0.649 | 12 | |
| Multiple-choice Video Question Answering | MLVU (test) | Accuracy73.8 | 10 | |
| Video Question Answering | MLVU 2024 (test) | Accuracy54.9 | 9 | |
| Video Understanding | MLVU | M-Aug68.6 | 8 | |
| Video Question Answering | MLVU 1.0 (test) | Accuracy67.8 | 6 | |
| Video Understanding Reasoning | MLVU | Accuracy0.661 | 6 | |
| Video Understanding | MLVU | Wall-time Speedup2.78 | 4 |
Showing 25 of 26 rows