Share your thoughts, 1 month free Claude Pro on usSee more

Over-refusal on XSTest

0Overrefusal Rate

Initial

Updated 6d ago

Evaluation Results

Method
Initial 2026.01	0	-	-	-	-	-
SafePath 2026.01	0	-	-	-	-	-
GPT-4o 2025.02	0.4	-	-	-	-	-
GPT-4o 2025.02	2.6	-	-	-	-	-
No Defense 2025.10	3	-	-	-	-	-
o1 2025.02	3.9	-	-	-	-	-
sonnet-4-thinking 2025.10	4	-	-	-	-	-
Vanilla 2025.10	4	-	-	-	-	-
ECSO 2025.10	4.8	-	-	-	-	-
WildJailbreak 2025.10	5	-	-	-	-	-
ASCoT (open) 2025.10	5	-	-	-	-	-
No Defense 2025.10	5	-	-	-	-	-
WildJailbreak 2025.10	5	-	-	-	-	-
Claude 3.5 Sonnet 2025.02	5.2	-	-	-	-	-
ASTRA 2025.10	5.4	-	-	-	-	-
o1 2025.02	5.7	-	-	-	-	-
o4-mini 2025.10	6	-	-	-	-	-
ASCoT 2025.10	6	-	-	-	-	-
No Defense 2025.10	6	-	-	-	-	-
WildJailbreak 2025.10	6	-	-	-	-	-
ASCoT (open) 2025.10	6	-	-	-	-	-
PAP Trained 2025.10	6	-	-	-	-	-
CAT 2025.10	7	-	-	-	-	-
ASCoT (closed) 2025.10	7	-	-	-	-	-
ASCoT (open) 2025.10	7	-	-	-	-	-
MoRAS 2025.10	7.2	-	-	-	-	-
Claude 3.5 Sonnet 2025.02	7.4	-	-	-	-	-
GPT-3.5-turbo 2025.02	7.8	-	-	-	-	-
GPT-3.5-turbo 2025.02	7.8	-	-	-	-	-
Refusal Training 2025.10	9	-	-	-	-	-
Refusal Training 2025.10	9	-	-	-	-	-
Refusal Training 2025.10	10	-	-	-	-	-
ASCoT (closed) 2025.10	10	-	-	-	-	-
PAP Trained 2025.10	10	-	-	-	-	-
PAP Trained 2025.10	12	-	-	-	-	-
ASCoT (closed) 2025.10	12	-	-	-	-	-
ETA 2025.10	15.6	-	-	-	-	-
CoCA 2025.10	24.8	-	-	-	-	-
LAT 2025.10	30	-	-	-	-	-
LAT* 2025.10	30	-	-	-	-	-
CAT 2025.10	32	-	-	-	-	-
LAT* 2025.10	34	-	-	-	-	-
SafeChain 2026.01	40	-	-	-	-	-
SafePath 2026.01	40	-	-	-	-	-
SafeChain 2026.01	40	-	-	-	-	-
THINKSAFE 2026.01	40	-	-	-	-	-
Initial 2026.01	40	-	-	-	-	-
SafePath 2026.01	40	-	-	-	-	-
CAT* 2025.10	40	-	-	-	-	-
LAT 2025.10	44	-	-	-	-	-
FigStep 2025.10	46	-	-	-	-	-
CAT* 2025.10	47	-	-	-	-	-
CAT* 2025.10	49	-	-	-	-	-
AutoSteer 2025.10	62.4	-	-	-	-	-
DPO_Whisperer 2025.05	80.67	-	-	-	-	-
LAT 2025.10	86	-	-	-	-	-
Mixtral (SFT_OG) 2025.05	87.6	-	-	-	-	-
Mixtral (SFT_DB) 2025.05	91.84	-	-	-	-	-
SFT_DB 2025.05	91.84	-	-	-	-	-
Qwen (SFT_DB) 2025.05	93.6	-	-	-	-	-
Qwen (SFT_OG) 2025.05	98	-	-	-	-	-
LAT* 2025.10	98	-	-	-	-	-
CAT 2025.10	98	-	-	-	-	-
Mixtral (Base) 2025.05	98.8	-	-	-	-	-
Qwen (Base) 2025.05	99.2	-	-	-	-	-
STAR-1 2026.01	120	-	-	-	-	-
THINKSAFE 2026.01	120	-	-	-	-	-
Initial 2026.01	120	-	-	-	-	-
SafeChain 2026.01	160	-	-	-	-	-
THINKSAFE 2026.01	160	-	-	-	-	-
STAR-1 2026.01	240	-	-	-	-	-
SafeKey 2026.01	360	-	-	-	-	-
SafeKey 2026.01	720	-	-	-	-	-
SafeKey 2026.01	800	-	-	-	-	-
DirectRefusal 2026.01	840	-	-	-	-	-
STAR-1 2026.01	1,200	-	-	-	-	-
DirectRefusal 2026.01	4,360	-	-	-	-	-
DirectRefusal 2026.01	5,000	-	-	-	-	-
Original Weight Model 2025.04	-	85.78	63.81	-	-	-
TA 2025.04	-	80.22	72.96	-	-	-
NPO 2025.04	-	68.89	74.52	-	-	-
RMU 2025.04	-	78.44	71.85	-	-	-
CB 2025.04	-	86.89	77.34	-	-	-
R2D2 2025.04	-	67.56	72.67	-	-	-
CB 2025.04	-	86.22	73.62	-	-	-
REPBEND 2025.04	-	84.89	81.23	-	-	-
Original Weight Model 2025.04	-	85.11	73.9	-	-	-
TA 2025.04	-	80	70.71	-	-	-
NPO 2025.04	-	74.45	71.65	-	-	-
RMU 2025.04	-	76.89	72.9	-	-	-
CB 2025.04	-	84.44	81.61	-	-	-
CB 2025.04	-	85.78	75.87	-	-	-
REPBEND 2025.04	-	84.11	83.14	-	-	-
Original 2025.04	-	85.78	-	-	-	-
SFT 2025.04	-	78.89	-	-	-	-
DPO 2025.04	-	83.56	-	-	-	-
TA 2025.04	-	80.22	-	-	-	-
Safety Prompting 2025.04	-	81.77	-	-	-	-
WHP (W/O safe set) 2025.04	-	85.56	-	-	-	-
WHP 2025.04	-	80.44	-	-	-	-

Showing 100 of 150 rows