Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Video-MME

Benchmarks

Task NameDataset NameSOTA ResultTrend
Video UnderstandingVideo-MME without subtitles
Overall Score75
67
Long Video UnderstandingVideo-MME long 1.0
Accuracy (No Subs)67.4
45
Video UnderstandingVideo-MME (test)
Accuracy (No Subtitles)84.3
40
Long Video UnderstandingVideo-MME Overall
Accuracy87
39
Multi-modal Video EvaluationVideo-MME
Accuracy75
38
Long Video UnderstandingVideo-MME Long
Accuracy81.9
37
Video Question AnsweringVideo-MME Long Duration 1.0
Accuracy (w/o subtitles)67.4
34
Video UnderstandingVideo-MME Long
Accuracy (Long, wo Sub)67.4
32
Video Question AnsweringVideo-MME without subtitles
Accuracy (Overall)73.3
28
Video ReasoningVideo-MME
Short Query Performance72.4
24
Video SummarizationVideo-MME 900 videos
Overall Accuracy85.2
22
Video UnderstandingVideo-MME Sub (test)
Accuracy87.8
21
Video Question AnsweringVideo-MME Overall 1.0
Accuracy (No Subtitles)59
19
Video Question AnsweringVideo-MME Medium Duration 1.0
Accuracy (No Subtitles)58.1
18
Video Question AnsweringVideo-MME Short Duration 1.0
Accuracy (w/o subtitles)68.1
18
Long Video UnderstandingVideo-MME (full)
Overall Performance59.7
16
Video Understanding and ReasoningVideo-MME (test)
Overall Accuracy60.2
15
Audio-Visual UnderstandingVideo-MME
Score73.4
15
Long Video Question AnsweringVideo-MME w/o subtitles
Accuracy0.818
14
Video UnderstandingVideo-MME With Subtitles
Performance (Short)75.8
14
Multimodal Video ComprehensionVideo-MME
Average Sparsity0
14
Video Question AnsweringVideo-MME long overall durations
Acc (Long, -subs)66.46
13
Video UnderstandingVideo-MME w/o audio
Accuracy64.4
13
Long Video UnderstandingVideo-MME w/o sub (full)
Score (Long)81.2
13
Video Question AnsweringVideo-MME
Accuracy86.9
12
Showing 25 of 42 rows