VideoMME
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Understanding | VideoMME | Score (Overall)75 | 369 | |
| Video Question Answering | VideoMME | Accuracy85.1 | 254 | |
| Video Understanding | VideoMME | Overall Score100 | 222 | |
| Long Video Understanding | VideoMME | Accuracy89.5 | 97 | |
| Video Understanding | VideoMME | Accuracy (No Subtitles)67.4 | 67 | |
| Multi-modal Video Understanding | VideoMME | Accuracy84.3 | 64 | |
| Video Question Answering | VideoMME (test) | Short Length Accuracy89.8 | 61 | |
| Video Question Answering | VideoMME Medium | Accuracy72.9 | 53 | |
| Video Question-Answering | VideoMME wo sub | Accuracy88.6 | 51 | |
| Multi-modal Video Evaluation | VideoMME | Score71.9 | 50 | |
| Video Question Answering | VideoMME 16 (test) | Medium Length Score70.11 | 45 | |
| Video Understanding | VideoMME (test) | Overall Score86.9 | 45 | |
| Video Question Answering | VideoMME | VQA Score (wo subs)75 | 45 | |
| Video Question Answering | VideoMME Overall | Accuracy73.6 | 40 | |
| Video Understanding | VideoMME w/o sub | Score82.5 | 36 | |
| Video Understanding | VideoMME | Accuracy71.9 | 33 | |
| Video Question Answering | VideoMME | Short VQA Accuracy76 | 32 | |
| Video Understanding | VideoMME | Score (%)75 | 28 | |
| Offline Video Understanding | VideoMME v1 (test) | Accuracy77.2 | 27 | |
| Long Video Understanding | VideoMME Long split, 30-60 min | Accuracy65.3 | 27 | |
| Video Understanding | VideoMME Long | Score59 | 25 | |
| Video Understanding | VideoMME w/o sub. Overall | Overall Accuracy75 | 22 | |
| Long-video understanding | VideoMME Long (3~120 min) | Score72.1 | 22 | |
| General Multi-task Video Understanding | VideoMME w/o sub | Average Accuracy77.4 | 22 | |
| Video Understanding | VideoMME | Accuracy (Base)65.6 | 22 |
Showing 25 of 94 rows