Video-MME
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Video Understanding | Video-MME without subtitles | Overall Score84.8 | 145 | |
| General Video Understanding | Video-MME | Accuracy87.8 | 139 | |
| Long Video Understanding | Video-MME Long | Accuracy81.9 | 120 | |
| Video Understanding | Video-MME | Overall Score78.18 | 96 | |
| Video Understanding | Video-MME | Overall Score78.78 | 92 | |
| Video Understanding | Video-MME | Accuracy84.8 | 90 | |
| Long-video understanding | Video-MME | Overall Score88.6 | 90 | |
| Long Video Understanding | Video-MME Overall | Accuracy87 | 81 | |
| Video Question Answering | Video-MME without subtitles | Accuracy (Overall)75.4 | 81 | |
| Video Reasoning | Video-MME | Accuracy68.4 | 73 | |
| Video Question Answering | Video-MME Long | Accuracy82 | 71 | |
| Multi-modal Video Evaluation | Video-MME | Accuracy75 | 57 | |
| Video Understanding | Video-MME v1.0 (test) | Score (Short)72.4 | 56 | |
| Long Video Understanding | Video-MME (full) | Overall Performance66.4 | 51 | |
| Video Understanding | Video-MME (test) | Accuracy88.6 | 51 | |
| Video Understanding | Video-MME | Score (Overall)76.3 | 49 | |
| Video Question Answering | Video-MME | Accuracy (Average, wo/ Subtitle)82.8 | 48 | |
| Video Understanding | Video-MME w/o sub. | Accuracy76.6 | 46 | |
| Long Video Understanding | Video-MME long 1.0 | Accuracy (No Subs)67.4 | 45 | |
| Video Multimodal Understanding | Video MME | Score61.9 | 43 | |
| Long Video QA | Video-MME | Average Score84.3 | 41 | |
| Video Question Answering | Video-MME Long Duration 1.0 | Accuracy (w/o subtitles)67.4 | 34 | |
| Video Question Answering | Video-MME | Accuracy86.9 | 32 | |
| Video Understanding | Video-MME Long | Accuracy (Long, wo Sub)67.4 | 32 | |
| Long Video Understanding | Video MME w/o sub (long) | Accuracy71.4 | 30 |
Showing 25 of 107 rows