VideoMME
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Understanding | VideoMME | Score (Long)67.4 | 248 | |
| Video Understanding | VideoMME | Overall Score100 | 222 | |
| Video Question Answering | VideoMME | Accuracy85.1 | 210 | |
| Video Understanding | VideoMME | Accuracy (No Subtitles)65.1 | 60 | |
| Video Question-Answering | VideoMME wo sub | Accuracy88.6 | 51 | |
| Multi-modal Video Understanding | VideoMME | Accuracy84.3 | 50 | |
| Video Question Answering | VideoMME 16 (test) | Medium Length Score70.11 | 45 | |
| Multi-modal Video Evaluation | VideoMME | Score65.1 | 42 | |
| Long Video Understanding | VideoMME | Accuracy81.3 | 40 | |
| Video Understanding | VideoMME (test) | Overall Score86.9 | 34 | |
| Video Question Answering | VideoMME | VQA Score (wo subs)75 | 31 | |
| Video Question Answering | VideoMME Medium | Accuracy61.9 | 27 | |
| Video Understanding | VideoMME Long | Score59 | 25 | |
| General Multi-task Video Understanding | VideoMME w/o sub | Average Accuracy77.4 | 22 | |
| Video Understanding | VideoMME | Accuracy (Base)65.6 | 22 | |
| Video Understanding | VideoMME v1.0 (test) | Score60.8 | 21 | |
| Video Understanding | VideoMME | Wall-time Speedup3.27 | 21 | |
| Multi-choice Video Question Answering | VideoMME | Accuracy (no subs)75 | 21 | |
| Video Question Answering | VideoMME KFS-Bench full set | Accuracy68.4 | 20 | |
| Video Question Answering | VideoMME (test) | Overall Accuracy78.6 | 19 | |
| Video Understanding | VideoMME w/o sub | Score75 | 18 | |
| Long-Video Understanding | VideoMME 1~60m, w/o subtitles | Score75 | 18 | |
| Video Question Answering | VideoMME (long split) | Accuracy67.4 | 18 | |
| Video Question Answering | VideoMME | Accuracy (Overall, w/o Subtitles)77.4 | 16 | |
| Long Video Understanding | VideoMME Long w/o sub | Accuracy70.3 | 16 |
Showing 25 of 64 rows