Share your thoughts, 1 month free Claude Pro on usSee more

Over Refusal on Over Refusal scenario

98.7ASR (Attacked)

OLMo-2-7B

Updated 12d ago

Evaluation Results

Method	Links
OLMo-2-7B 2025.10		98.7	4.6
Qwen2.5-7B 2025.10		98.4	1.7
Llama3.1-8B 2025.10		97.3	2.9
OLMo-2-7B 2025.10		97.2	4.1
Qwen2.5-7B 2025.10		96.3	4.1
Llama3.1-8B 2025.10		95.5	0.3
Qwen2.5-7B 2025.10		93.9	0.3
Qwen2.5-7B 2025.10		93.7	0.8
Llama3.1-8B 2025.10		93	0.8
OLMo-2-7B 2025.10		92.7	2.1
OLMo-2-7B 2025.10		91.1	2.1
OLMo-2-7B 2025.10		78.8	2.6
OLMo-2-7B 2025.10		78.7	6.9
Llama3.1-8B 2025.10		78.3	2.7
Llama3.1-8B 2025.10		70.4	0.3
Qwen2.5-7B 2025.10		67.8	1.1
Llama3.1-8B 2025.10		63.2	1.8
Qwen2.5-7B 2025.10		51.3	0.9
OLMo-2-7B 2025.10		47.9	8.7
Qwen2.5-7B 2025.10		40.9	2.8
Llama3.1-8B 2025.10		21.4	2.9
OLMo-2-7B 2025.10		2.1	2.5
Qwen2.5-7B 2025.10		1.1	0.4
Llama3.1-8B 2025.10		0.5	0.5