Share your thoughts, 1 month free Claude Pro on usSee more

Multimodal Evaluation Consistency on MLLM-as-a-Judge

39.6CO Score

GPT-4o

Updated 1mo ago

Evaluation Results

Method
GPT-4o 2025.06	39.6	45.2	34.1	46.4	46	56.4	40.8	57.3	58.9	30.5	26.2	56.9	42.1	34.2	-
Gemini-2.5-Pro 2025.06	38.9	40.7	43.4	56.8	49.5	59.6	34	60.7	50.1	22.7	-0.5	40	20	37.3	-
Gemini-2.5-Pro 2025.06	38.9	40.7	43.4	56.8	49.5	59.6	34	60.7	50.1	22.7	-0.5	40	20	37.3	-
LLaVA-Critic 2025.06	38.2	45	10.3	31.6	35.6	37.8	17.9	42.1	32.2	24.6	30.1	26.9	39.5	27.2	-
LLaVA-Critic(72B) 2025.06	37	36	8.6	43.1	45.7	46.8	20.7	53	44	26.5	14.7	35.2	27.1	42.2	-
LLaVA-Critic 2025.06	33.3	46.3	14.6	45.2	47.4	55.9	39.6	54.5	48.8	27.3	25.9	33.4	40.3	37.4	-
Minos 2025.06	32.8	41.1	25.1	54.1	51.3	58.3	41.4	62.1	47.1	24.3	23.6	50.7	28.7	39.8	-
Minos(8B) 2025.06	32.8	41.1	25.1	54.1	51.3	58.3	41.4	62.1	47.1	24.3	23.6	50.7	28.7	39.8	-
GPT-4o 2025.06	30.1	33	22.2	44.3	44.3	52.8	32.8	57.4	40.1	22.4	6.5	36.2	29.3	31.2	-
UnifiedReward_Q 2025.06	29.3	35.1	22.3	44	46.6	43.4	29	55.3	37.9	25	17.2	43.1	30.6	33.6	-
UnifiedReward_Q(8B) 2025.06	29.3	35.1	22.3	44	46.6	43.4	29	55.3	37.9	25	17.2	43.1	30.6	33.6	-
Prometheus-V 2025.06	28.9	34.2	10.6	17.2	18.2	21.4	20.9	22.4	22.6	22.8	8.9	17.4	36.8	15.7	-
LLaVA-OV(72B) 2025.06	27.5	12.4	9.7	20.1	31.6	31.3	8.9	26	35.1	25.6	31.5	27.2	26.3	37.2	-
LLaVA-OV 2025.06	26.4	39	4.6	26.2	35.8	32.7	19.5	29	41.5	14.4	35.9	26.7	44.4	25.3	-
Qwen3-VL 2025.06	26.4	37.3	28.5	57	49	51.2	43.2	56.4	43.3	27.2	-0.1	39.6	24.8	29.3	-
Qwen3-VL(8B) 2025.06	26.4	37.3	28.5	57	49	51.2	43.2	56.4	43.3	27.2	-0.1	39.6	24.8	29.3	-
LLaVA-Critic(7B) 2025.06	25.9	34.3	15.1	37.9	28.5	38.4	17.6	32	34.1	20.1	23.1	15.6	39.2	32.3	-
UnifiedReward_L 2025.06	25	37.8	19.6	34.4	40.5	44	22.6	39.5	35.9	22.9	22	36.9	32.4	21.8	-
UnifiedReward_L(7B) 2025.06	25	37.8	19.6	34.4	40.5	44	22.6	39.5	35.9	22.9	22	36.9	32.4	21.8	-
LLaVA-OV 2025.06	22.4	2.4	6.3	18.9	9.7	26.5	-13.5	27.4	22.7	8.1	3	26.1	24.9	26.2	-
LLaVA-OV(7B) 2025.06	20.6	19.4	13.6	21.1	14.8	28.2	4.9	30.3	29.6	16.9	34.2	13.5	42.3	39.1	-
Prometheus-V(7B) 2025.06	13.4	10.3	12.2	15.4	13.7	21.3	19.7	13.5	14.2	16.1	18.1	11.9	36.3	15.8	-