Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Video-MME

Benchmarks

Task NameDataset NameSOTA ResultTrend
Video UnderstandingVideo-MME
Overall Score78.18
96
Video UnderstandingVideo-MME
Overall Score78.78
92
Video UnderstandingVideo-MME without subtitles
Overall Score75
89
Video UnderstandingVideo-MME v1.0 (test)
Score (Short)72.4
56
Video UnderstandingVideo-MME (test)
Accuracy88.6
51
Video Question AnsweringVideo-MME
Accuracy (Average, wo/ Subtitle)82.8
48
Long Video UnderstandingVideo-MME Long
Accuracy81.9
46
Long Video UnderstandingVideo-MME long 1.0
Accuracy (No Subs)67.4
45
Long Video QAVideo-MME
Average Score84.3
41
Long Video UnderstandingVideo-MME Overall
Accuracy87
39
Video ReasoningVideo-MME
Overall Performance73.1
39
Multi-modal Video EvaluationVideo-MME
Accuracy75
38
Video Question AnsweringVideo-MME Long
Accuracy82
36
Video Question AnsweringVideo-MME without subtitles
Accuracy (Overall)73.3
34
Video Question AnsweringVideo-MME Long Duration 1.0
Accuracy (w/o subtitles)67.4
34
Video Multimodal UnderstandingVideo MME
Score61.9
33
Video UnderstandingVideo-MME Long
Accuracy (Long, wo Sub)67.4
32
Long Video UnderstandingVideo MME w/o sub (long)
Accuracy71.4
30
Long-video understandingVideo-MME
Overall Score84.3
30
Long-form Video Multimodal EvaluationVideo-MME Long
Video-MME-Long Score56.6
24
Long Video UnderstandingVideo-MME (w/o sub.) Overall 1010s
Accuracy75
22
Multimodal Video EvaluationVideo-MME Sub (test)
Accuracy87.8
22
Multimodal Video EvaluationVideo-MME (test)
Accuracy88.6
22
Long Video Question AnsweringVideo-MME
Average Accuracy (w/o subs)75.7
22
Video SummarizationVideo-MME 900 videos
Overall Accuracy85.2
22
Showing 25 of 75 rows