Share your thoughts, 1 month free Claude Pro on usSee more

Video Understanding on VideoMME (Subtitle Accuracy)

65.1Accuracy (No Subtitles)

Vanilla

Updated 10d ago

Evaluation Results

Method
Vanilla 2024.12	65.1	71.6	100	100	-	-
LLaVA-Video 2026.04	64.1	71.4	-	-	-	-
iLLaVA 2024.12	63	69.9	97.7	96.7	-	-
iLLaVA 2024.12	62.3	68.6	96.6	95.7	-	-
HieraVid 2026.04	62.3	70.8	-	-	-	-
VisionZip 2024.12	62	69.2	96.6	95.3	-	-
Framefusion (ICCV2025) 2026.04	61.9	70.1	-	-	-	-
HieraVid 2026.04	61.8	66.3	-	-	-	-
FasterVLM 2024.12	61.6	68.6	95.8	94.7	-	-
PruneVid (ACL2025) 2026.04	61.5	69.6	-	-	-	-
PyramidDrop 2024.12	61.4	68.6	95.8	94.5	-	-
FasterVLM 2024.12	60.9	67.8	94.8	93.8	-	-
VisionZip 2024.12	60.9	67.9	94.9	93.6	-	-
PruMerge (ICCV2025) 2026.04	60.9	67.7	-	-	-	-
SparseVLM 2024.12	60.3	67.1	93.7	92.9	-	-
PyramidDrop 2024.12	60.2	67.6	94.5	93.2	-	-
SparseVLM 2024.12	59.6	66.4	92.8	91.6	-	-
Framefusion (ICCV2025) 2026.04	59.5	66.2	-	-	-	-
FastV (ECCV2024) 2026.04	59.3	66.7	-	-	-	-
PruneVid (ACL2025) 2026.04	59.3	66.2	-	-	-	-
HieraVid 2026.04	58.8	63.2	-	-	-	-
LLaVA-OneVision 2026.04	58.5	61.9	-	-	-	-
HieraVid 2026.04	58	61.6	-	-	-	-
PruneVid (ACL2025) 2026.04	57.7	62.8	-	-	-	-
Vanilla 2026.04	57.6	61	-	-	59.3	100
Framefusion 2026.04	57.5	61.1	-	-	-	-
PruneVid 2026.04	57.4	59.9	-	-	-	-
Framefusion (ICCV2025) 2026.04	57.3	61.7	-	-	-	-
HieraVid 2026.04	57.1	60.7	-	-	-	-
PruMerge 2026.04	56.6	58.8	-	-	-	-
PruMerge (ICCV2025) 2026.04	56.5	62.6	-	-	-	-
FastV 2026.04	56.3	58.1	-	-	-	-
HieraVid 2026.04	56.1	60.1	-	-	58.1	98
PruneVid 2026.04	55.8	58.4	-	-	-	-
Framefusion 2026.04	55.7	60.1	-	-	-	-
StreamingLLM (ICLR2024) 2026.04	55.5	65.7	-	-	-	-
FastV (ECCV2024) 2026.04	55.4	61	-	-	-	-
Framefusion 2026.04	55.4	59.7	-	-	57.6	97
HieraVid 2026.04	55.4	59.7	-	-	-	-
PruMerge 2026.04	54.2	56.5	-	-	-	-
PruneVid 2026.04	53.8	57.3	-	-	55.6	93.7
FastV 2026.04	53.7	57.1	-	-	-	-
PruneVid 2026.04	53.7	57.5	-	-	-	-
Framefusion 2026.04	53.5	56.5	-	-	-	-
StreamingLLM 2026.04	52	54.9	-	-	-	-
PruMerge 2026.04	51.9	54.7	-	-	-	-
FastV 2026.04	51.7	55.1	-	-	-	-
StreamingLLM (ICLR2024) 2026.04	51.6	61	-	-	-	-
PruMerge (ICCV2025) 2026.04	50	55	-	-	-	-
FastV (ECCV2024) 2026.04	49.6	52.5	-	-	-	-
StreamingLLM 2026.04	49.5	52.7	-	-	-	-
StreamingLLM 2026.04	48.8	51.2	-	-	-	-
StreamingLLM (ICLR2024) 2026.04	45.8	53.9	-	-	-	-
DEViL 2025.12	0.577	0.624	-	-	-	-
TimeMarker 2025.12	0.573	-	-	-	-	-
VideoLLaMA2-7B 2025.12	0.479	-	-	-	-	-
VideoChat2-7B 2025.12	0.423	0.546	-	-	-	-
Video-LLaVA-7B 2025.12	0.399	0.416	-	-	-	-
TimeChat-7B 2025.12	0.347	-	-	-	-	-
LLaVA-NeXT-Video-7B 2025.12	0.337	-	-	-	-	-
GPT-4o 2026.04	-	-	-	-	67.9	-
LLaVA-OneVision-7B 2026.04	-	-	-	-	58.2	-
LLaVA-Video-7B 2026.04	-	-	-	-	63.7	-
VideoLLaMA 3-7B 2026.04	-	-	-	-	61	-
MiniCPM-V 2.6-8B 2026.04	-	-	-	-	59.7	-
mPLUG-Owl3-8B 2026.04	-	-	-	-	53.5	-
InternVL2.5-8B 2026.04	-	-	-	-	63.7	-
Qwen2.5-VL-7B 2026.04	-	-	-	-	65.1	-
Video-R1 2026.04	-	-	-	-	59.3	-
STAR-R1 2026.04	-	-	-	-	56.6	-
TinyLLaVA-Video-R1 2026.04	-	-	-	-	46.6	-
VideoChat-R1 2026.04	-	-	-	-	58.8	-
VideoChat-R1.5 2026.04	-	-	-	-	67.1	-
VideoRFT 2026.04	-	-	-	-	59.8	-
MOSS-ChatV 2026.04	-	-	-	-	60	-
RLER 2026.04	-	-	-	-	68.5	-
Qwen2.5-VL-7B 2026.04	-	-	-	-	58.2	-
TW-GRPO 2026.04	-	-	-	-	58.2	-
LongVILA-R1-7B 2026.04	-	-	-	-	55.5	-
Video-RTS 2026.04	-	-	-	-	58.7	-
Qwen2.5-VL-7B-SFT 2026.04	-	-	-	-	58.2	-
Video-R1 2026.04	-	-	-	-	56.9	-
VidGround 2026.04	-	-	-	-	58.7	-
Qwen2.5-VL-7B 2026.04	-	-	-	-	60.7	-
TW-GRPO 2026.04	-	-	-	-	61.2	-
LongVILA-R1-7B 2026.04	-	-	-	-	60.2	-
Video-RTS 2026.04	-	-	-	-	61.3	-
Qwen2.5-VL-7B-SFT 2026.04	-	-	-	-	60.7	-
Video-R1 2026.04	-	-	-	-	60.2	-
VidGround 2026.04	-	-	-	-	61.5	-
Qwen2.5-VL-7B 2026.04	-	-	-	-	62.3	-
TW-GRPO 2026.04	-	-	-	-	62.7	-
LongVILA-R1-7B 2026.04	-	-	-	-	61.6	-
Video-RTS 2026.04	-	-	-	-	62.9	-
Qwen2.5-VL-7B-SFT 2026.04	-	-	-	-	62.2	-
Video-R1 2026.04	-	-	-	-	61.2	-
VidGround 2026.04	-	-	-	-	63.4	-