Share your thoughts, 1 month free Claude Pro on usSee more

Over-refusal Evaluation on XSTest

100Evaluation Score (avg@4)

Safechain

Updated 22d ago

Evaluation Results

Method	Links
Safechain 2026.05		100
Safechain 2026.05		100
Safechain 2026.05		99.6
SAGE 2026.06		99.6
Base 2026.05		98.8
DAPO 2026.05		98.5
Self-ReSET 2026.05		98.4
Self-ReSET 2026.05		96.4
Base 2026.05		96.4
Self-ReSET 2026.05		96.4
RECAP 2026.05		96.2
DAPO 2026.05		95.2
RECAP 2026.05		94.4
RECAP 2026.05		94
Base 2026.05		91.2
DAPO 2026.05		91.2
STAR-1 2026.05		87.6
STAR-1 2026.05		85.2
STAR-1 2026.05		73.6
SFT mix50/1000/aug 2026.05		66.8
SAGE 2026.06		61.2
SecDecoding 2026.06		42
SFT mix50/1000/aug 2026.05		30.4
Self-Reminder 2026.05		25.6
SafeDecoding 2026.06		20
PROACT 2026.06		19.7
THRD 2026.06		18.8
PROACT 2026.06		16.8
No Defense 2026.06		16
Specreason+RPO 2026.06		16
SafeDecoding 2026.06		16
SFT mix50/1000/aug 2026.05		13.6
THRD 2026.06		12.4
RPO 2026.06		12
SafeSpec 2026.06		12
Llama-Guard-3 2026.05		12
baseline 2026.05		11.2
DPO mix50/1000/aug 2026.05		10.8
SafeSpec 2026.06		10
No Defense 2026.06		9.6
SmoothLLM 2026.05		9.6
SFT mix50/200/aug 2026.05		8.4
Specreason 2026.06		8
SR 2026.06		8
Specreason+SR 2026.06		8
Llama-Guard-3 2026.05		8
baseline 2026.05		8
DPO mix50/1000/aug 2026.05		8
baseline 2026.05		7.2
DPO mix50/1000/aug 2026.05		7.2
SmoothLLM 2026.05		7.2
SFT mix0/200/clean 2026.05		6.4
baseline 2026.05		6
DPO mix50/1000/aug 2026.05		5.6
Spec-Decoding 2026.06		5
RPO 2026.06		5
SmoothLLM 2026.05		4.8
DPO mix50/2368/aug ext. pool⋄ 2026.05		4.5
baseline 2026.05		4.4
Specreason+SR 2026.06		4
SR 2026.06		4
Specreason+RPO 2026.06		4
SmoothLLM 2026.05		4
DPO mix50/1000/clean 2026.05		3.6
No Defense 2026.06		3
SecDecoding 2026.06		3
SmoothLLM 2026.05		2.4
No Defense 2026.06		1
Spec-Decoding 2026.06		1
Specreason 2026.06		0