Share your thoughts, 1 month free Claude Pro on usSee more

Compositional Reasoning on Compositional Reasoning Suite Aggregated

93.1Sugarcrepe Score

LLaVA-7B†

Updated 9d ago

Evaluation Results

Method	Links
LLaVA-7B† 2025.10		93.1	-	82.9	68.3	68.2	78.1
MiniCPM-V 2025.10		90	-	70.6	65.3	55.3	70.3
Qwen2.5-VL-7B 2025.10		88.5	-	77.4	68.4	75.3	77.4
LLaVA-7B† 2025.10		87.3	-	78.5	65.7	58.4	72.5
Deepseek-VL-7B 2025.10		86.2	-	78.8	66.2	61.7	73.2
LLaVA-4B† (Teacher) 2025.10		83	-	75.5	64.8	57.8	70.3
CompoDistill-2B 2025.10		82.9	-	69.4	63.3	51.2	66.7
CogVLM-7B 2025.10		81.8	-	63.2	64.5	57.2	66.7
MoE-LLaVA-2B 2025.10		80.8	-	62.5	62	50.6	63.9
MoE-LLaVA-3B 2025.10		80.5	-	70.4	64.4	54.1	67.3
Imp-3B 2025.10		78.1	-	61.1	59.5	38.3	59.3
LLaVADI-2B† 2025.10		76.9	-	61.5	54.1	48.4	60.2
LLaVA-MoD-2B† 2025.10		76.9	-	63.8	60.3	49.4	62.6
Bunny-3B 2025.10		75.8	-	67.2	59.7	53.1	64
LLaVA-KD-2B 2025.10		75.3	-	63.6	57.9	49.3	61.5
LLaVA-2B (SFT model) 2025.10		73.3	-	63.9	58.5	47.2	60.7
Imp-2B 2025.10		71	-	59.6	58.5	51.1	60.1
MobileVLM-1.7B 2025.10		69.8	-	64	53.8	43.4	57.7
Bunny-2B 2025.10		68.6	-	66.5	57.8	48.5	60.3
MobileVLM-3B 2025.10		67.9	-	72.1	57.2	43.9	60.3
MiniGemini-2B 2025.10		67	-	62	58	50.5	59.4
Qwen2.5-VL-3B 2025.10		57.1	-	65.9	67	63.5	63.4
Deepseek-VL-1.3B 2025.10		36.6	-	38.6	39.9	60.8	44
LaCLIP 2024.11		-	23.65	-	-	-	-
LaCLIP + HN 2024.11		-	20.15	-	-	-	-
NegCLIP 2024.11		-	23.09	-	-	-	-
NegCLIP++ 2024.11		-	23.2	-	-	-	-
TripletCLIP 2024.11		-	25.11	-	-	-	-
TripletCLIP++ 2024.11		-	26.64	-	-	-	-
LaCLIP 2024.11		-	25.96	-	-	-	-
NegCLIP 2024.11		-	25.51	-	-	-	-
NegCLIP++ 2024.11		-	27.48	-	-	-	-
TripletCLIP 2024.11		-	28.06	-	-	-	-