Our new X account is live! Follow @wizwand_team for updates

Video Generation Evaluation on Video-ChatGPT benchmark

61.6Correctness Score

CAT

Updated 4d ago

Evaluation Results

Method
CAT 2024.03	61.6	62	69.8	56.2	57.8	-
LLAMA-VID 2024.03	61.4	61	72	51.6	52.6	-
VideoChat2 2024.03	60.4	57.8	70.2	53.2	56.2	-
LLAMA-VID 2024.03	59.2	60	70.6	49.2	50.2	-
Chat-UniVi 2024.03	57.8	58.2	69.2	57.8	56.2	-
VISTA-LLAMA 2024.03	48.8	52.8	63.6	45.2	46.2	-
Video-ChatGPT 2024.03	48	50.4	52.4	39.6	47.4	-
VideoChat 2024.03	44.6	50	50.6	38.8	44.8	-
LLaMA-Adapter 2024.03	40.6	46.4	46	39.6	43	-
Video-LLaMA 2024.03	39.2	43.6	43.2	36.4	35.8	-
GPT-4V 2024.02	3.85	3.45	3.84	3.63	2.8	-
LLaVA-NeXT-Video-DPO 2024.11	3.81	3.55	4.24	3.14	4.12	3.77
LLaVA-NeXT-Video-DPO 2024.11	3.64	3.45	4.17	2.95	4.08	3.66
VLM-RLAIF 2024.02	3.63	3.25	4	3.23	3.32	-
TS-LLAVA 2024.11	3.55	3.03	3.86	2.77	3.69	3.38
LLaVA-NeXT-Video 2024.11	3.48	3.37	3.95	2.64	3.28	3.34
SF-LLAVA 2024.11	3.48	2.96	3.84	2.77	3.57	3.32
LLaVA-NeXT-Video 2024.11	3.39	3.29	3.92	2.6	3.12	3.26
LLaVA-NeXT-Image 2024.11	3.29	3.23	3.83	2.51	3.47	3.27
VideoGPT+ 2024.11	3.27	3.18	3.74	2.83	3.39	3.28
ST-LLM 2024.03	3.23	3.05	3.74	2.93	2.81	3.15
IG-VLM 2024.11	3.21	2.87	3.54	2.51	3.34	3.09
TS-LLaVA 2024.11	3.18	2.82	3.58	2.53	3.34	3.09
Video-LLaMA2 2024.11	3.16	3.08	3.69	2.56	3.14	3.13
IG-VLM 2024.11	3.11	2.78	3.51	2.44	3.29	3.03
SF-LLaVA 2024.11	3.09	2.7	3.57	2.52	3.35	3.04
Video-LLaMA2 2024.11	3.08	3.11	3.64	2.67	3.26	3.15
LLAMA-VID 2023.11	3.07	3.05	3.6	2.58	2.63	-
LLaMA-VID-13B 2024.03	3.07	3.05	3.6	2.58	2.63	2.99
LLaMA-VID 2024.02	3.07	3.05	3.6	2.58	2.63	-
LLaVA-NeXT-Image 2024.11	3.05	3.12	3.68	2.37	3.16	3.07
VideoChat2 2024.03	3.02	2.88	3.51	2.66	2.81	2.98
VideoChat2 2024.02	3.02	2.88	3.51	2.66	2.81	-
VideoChat2 2024.11	3.02	2.88	3.51	2.66	2.81	2.98
LLAMA-VID 2023.11	2.96	3	3.53	2.46	2.51	-
LLaMA-VID-7B 2024.03	2.96	3	3.53	2.46	2.51	2.89
LLaMA-VID 2024.02	2.96	3	3.53	2.46	2.51	-
LLaMA-VID 2024.11	2.96	3	3.53	2.46	2.51	2.89
LITA 2024.03	2.94	2.98	3.43	2.68	3.19	3.04
Chat-UniVi 2024.03	2.89	2.91	3.46	2.89	2.81	2.99
Video-LLaVA 2024.02	2.84	2.86	3.44	2.46	2.57	-
VLM-SFT 2024.02	2.79	2.82	3.37	2.28	2.49	-
VTimeLLM 2023.11	2.78	3.1	3.4	2.49	2.47	2.85
VTimeLLM 2024.03	2.78	3.1	3.4	2.49	2.47	2.85
VTimeLLM 2024.02	2.78	3.1	3.4	2.49	2.47	-
MovieChat 2024.11	2.76	2.93	3.01	2.24	2.42	2.67
BT-Adapter 2023.11	2.68	2.69	3.27	2.34	2.46	-
BT-Adapter 2023.11	2.68	2.69	3.27	2.34	2.46	2.69
BT-Adapter 2024.03	2.68	2.69	3.27	2.34	2.46	2.69
BT-Adapter 2024.02	2.68	2.69	3.27	2.34	2.46	-
Video-ChatGPT 2024.11	2.5	2.57	2.69	2.16	2.2	2.42
Vista-LLaMA 2024.11	2.44	2.64	3.18	2.26	2.31	2.57
Valley 2024.02	2.43	2.13	2.86	2.04	2.45	-
Video-ChatGPT 2023.11	2.4	2.52	2.62	1.98	2.37	-
VideoChatGPT 2023.11	2.4	2.52	2.62	1.98	2.37	2.38
Video-ChatGPT 2024.03	2.4	2.52	2.62	1.98	2.37	2.38
VideoChatGPT 2024.03	2.4	2.52	2.62	1.98	2.37	2.38
Video-ChatGPT 2024.02	2.4	2.52	2.62	1.98	2.37	-
Video-LLaMA-v2 2024.03	2.36	2.42	2.74	1.83	2.12	2.29
VideoChat 2023.11	2.23	2.5	2.53	1.94	2.24	-
VideoChat 2023.11	2.23	2.5	2.53	1.94	2.24	2.29
VideoChat 2024.03	2.23	2.5	2.53	1.94	2.24	2.29
VideoChat 2024.03	2.23	2.5	2.53	1.94	2.24	2.29
VideoChat 2024.02	2.23	2.5	2.53	1.94	2.24	-
VideoChat 2024.11	2.23	2.5	2.53	1.94	2.24	2.29
LLaMA-Adapter 2023.11	2.03	2.32	2.3	1.98	2.15	-
LLaMA-Adapter 2023.11	2.03	2.32	2.3	1.98	2.15	2.16
LLaMA-Adapter 2024.03	2.03	2.32	2.3	1.98	2.15	2.16
LLaMA-Adapter 2024.03	2.03	2.32	2.3	1.98	2.15	2.16
LLaMA-Adapter 2024.02	2.03	2.32	2.3	1.98	2.15	-
VideoLLaMA 2023.11	1.96	2.18	2.16	1.82	1.79	-
VideoLLaMA 2023.11	1.96	2.18	2.16	1.82	1.79	1.98
Video-LLaMA 2024.03	1.96	2.18	2.16	1.82	1.79	1.98
VideoLLaMA 2024.03	1.96	2.18	2.16	1.82	1.79	1.98
VideoLLaMA 2024.02	1.96	2.18	2.16	1.82	1.79	-
Video-LLaMA 2024.11	1.96	2.18	2.16	1.82	1.79	1.98