MVBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Understanding | MVBench | Accuracy100 | 635 | |
| Video Understanding | MVBench (test) | Accuracy100.3 | 201 | |
| Video Question Answering | MVBench | Accuracy81.3 | 90 | |
| Multi-modal Video Understanding | MVBench | Score79.4 | 84 | |
| Multi-choice Video Question Answering | MVBench | Avg Accuracy67.6 | 73 | |
| Video Question Answering | MVBench | Accuracy77.1 | 72 | |
| Video Reasoning | MVBench | MVBench Score84.3 | 56 | |
| Video Question Answering | MVBench (test) | Accuracy73.6 | 45 | |
| Video Understanding | MVBench | MVBench Score67.5 | 39 | |
| General Video Understanding | MVBench Overall | Accuracy86.03 | 39 | |
| Adversarial Attack | MVBench | ASR83.84 | 37 | |
| Short video understanding | MVBench | Accuracy76.4 | 28 | |
| Video Understanding | MVBench zero-shot | Accuracy69.7 | 25 | |
| Video Question Answering | MVBench 1.0 (test) | AS Score77 | 25 | |
| Fine-grained Video Understanding | MVBench | Accuracy73.8 | 22 | |
| Video Question Answering | MVBench | Average Score61.2 | 20 | |
| Video understanding | MVBench (val) | Accuracy54.9 | 20 | |
| Multiple-choice question answering | MVBench | Accuracy58.54 | 15 | |
| Video Generalization | MVBench | MVBench Score72.3 | 14 | |
| Exocentric Video Understanding | MVBench | Score73.8 | 13 | |
| Text Generation | MVBench | MVBench Score75.4 | 12 | |
| Event-level Video Understanding | MVBench | Average Accuracy58.3 | 12 | |
| General spatiotemporal perception | MVbench | Score49 | 11 | |
| Video Question Answering | MVBench | Overall Score59.13 | 10 | |
| Video Question Answering | MVBench | Accuracy72 | 10 |
Showing 25 of 37 rows