Share your thoughts, 1 month free Claude Pro on usSee more

Multimodal Safety Evaluation on SPA-VL

91.89Safety Score

SaFeR-ToolKit (+ SFT+DPO+GRPO) [3B]

Updated 1mo ago

Evaluation Results

Method	Links
SaFeR-ToolKit (+ SFT+DPO+GRPO) [3B] 2026.03		91.89	85.09	87.36
SaFeR-ToolKit (+ SFT+DPO+GRPO) [7B] 2026.03		91.89	89.25	90.94
Qwen2.5-VL-7B 2026.03		76.42	73.21	40
Qwen2.5-VL-3B 2026.03		46.6	54.53	11.89
Qwen2.5-VL-3B + SFT+DPO+GRPO 2026.03		2.81	2.8	2.84
Qwen2.5-VL-7B + TIS 2026.03		2.8	2.68	2.72
Qwen2.5-VL-7B + SPA-VL 2026.03		2.8	2.7	2.28
Qwen2.5-VL-3B + SFT+GRPO 2026.03		2.78	2.7	2.82
Qwen2.5-VL-7B + SFT+DPO+GRPO 2026.03		2.77	2.87	2.88
Qwen2.5-VL-7B + SFT+GRPO 2026.03		2.75	2.84	2.88
Qwen2.5-VL-7B + SaFeR-VLM 2026.03		2.73	2.4	2.46
Qwen2.5-VL-3B + SPA-VL 2026.03		2.72	2.65	2.2
Qwen2.5-VL-7B + SFT+DPO 2026.03		2.72	2.72	2.86
Qwen2.5-VL-7B + SFT 2026.03		2.68	2.65	2.78
Qwen2.5-VL-3B + TIS 2026.03		2.62	2.35	2.5
Qwen2.5-VL-3B + SaFeR-VLM 2026.03		2.62	2.31	2.27
Qwen2.5-VL-3B + SFT+DPO 2026.03		2.6	2.54	2.67
Qwen2.5-VL-3B + SFT 2026.03		2.52	2.46	2.63
Qwen2.5-VL-7B + VLGuard 2026.03		2.49	1.67	1.8
Qwen2.5-VL-7B + SIA 2026.03		2.46	2.01	1.94
Qwen2.5-VL-3B + VLGuard 2026.03		2.43	1.63	1.74
Qwen2.5-VL-7B + ECSO 2026.03		2.37	2.64	2.27
Qwen2.5-VL-7B 2026.03		2.27	2.66	2.27
Qwen2.5-VL-3B + SIA 2026.03		1.7	1.63	1.38
Qwen2.5-VL-3B + ECSO 2026.03		1.39	2.16	1.66
Qwen2.5-VL-3B 2026.03		1.38	2.35	1.73