Share your thoughts, 1 month free Claude Pro on usSee more

Safety Evaluation (Detailed Attack Metrics) on HarmBench

95MD

SFT

Updated 1mo ago

Evaluation Results

Method	Links
SFT 2026.03		95	-	-	-	-	-	-	-	-	-	74
SFT 2026.03		93.5	-	-	-	-	-	-	-	-	-	81.5
Llama-3.2-3B + SFT 2026.03		89.5	-	-	-	-	-	-	-	-	-	73
Llama-3-8B 2026.03		87.5	-	-	-	-	-	-	-	-	-	23.5
Pythia-2.8B 2026.03		86	-	-	-	-	-	-	-	-	-	29.5
Llama-3.2-3B 2026.03		85	-	-	-	-	-	-	-	-	-	26.5
Pythia-2.8B + SFT 2026.03		81	-	-	-	-	-	-	-	-	-	28
Pythia-2.8B + cDPO 2026.03		76	-	-	-	-	-	-	-	-	-	14.5
Pythia-2.8B + IPO 2026.03		73	-	-	-	-	-	-	-	-	-	16.5
Llama-3.2-3B + cDPO 2026.03		65.5	-	-	-	-	-	-	-	-	-	15
Pythia-2.8B + DPO 2026.03		65	-	-	-	-	-	-	-	-	-	15
cDPO 2026.03		61	-	-	-	-	-	-	-	-	-	44
Pythia-2.8B + rDPO 2026.03		55.5	-	-	-	-	-	-	-	-	-	17
cDPO 2026.03		45.5	-	-	-	-	-	-	-	-	-	19.5
Qwen-2.5-7B 2026.03		44.5	-	-	-	-	-	-	-	-	-	27
DPO 2026.03		41	-	-	-	-	-	-	-	-	-	28.5
IPO 2026.03		33.5	-	-	-	-	-	-	-	-	-	17.5
DPO 2026.03		32.5	-	-	-	-	-	-	-	-	-	14
IPO 2026.03		30.5	-	-	-	-	-	-	-	-	-	14
Pythia-2.8B + wDPO 2026.03		21.5	-	-	-	-	-	-	-	-	-	1.5
Pythia-2.8B + Dr.DPO 2026.03		17.5	-	-	-	-	-	-	-	-	-	4.5
Llama-3.2-3B + IPO 2026.03		17	-	-	-	-	-	-	-	-	-	6
Llama-3.2-3B + DPO 2026.03		14	-	-	-	-	-	-	-	-	-	4
rDPO 2026.03		6	-	-	-	-	-	-	-	-	-	1.5
Llama-3.2-3B + rDPO 2026.03		3.5	-	-	-	-	-	-	-	-	-	1
rDPO 2026.03		3	-	-	-	-	-	-	-	-	-	1
Dr.DPO 2026.03		2	-	-	-	-	-	-	-	-	-	0
Llama-3.2-3B + Dr.DPO 2026.03		2	-	-	-	-	-	-	-	-	-	0.5
Llama-3.2-3B + wDPO 2026.03		2	-	-	-	-	-	-	-	-	-	0.5
wDPO 2026.03		1.5	-	-	-	-	-	-	-	-	-	0
Dr.DPO 2026.03		0.5	-	-	-	-	-	-	-	-	-	0
wDPO 2026.03		0	-	-	-	-	-	-	-	-	-	0
Base 2025.09		-	71	24.25	20	50.5	38.5	40.25	39	38.25	40.22	-
Post-hoc (LlamaGuard) 2025.09		-	100	47	51.25	60.5	98	88.5	72.25	85.5	75.38	-
STAIR-DPO 2025.09		-	90.75	52.25	31	77.25	96.25	62	57.5	75	67.75	-
WJ-SFT 2025.09		-	100	42.5	28.25	71.5	94	53	59	78	65.78	-
ReSA-SFT (Ours) 2025.09		-	100	84.5	64.5	96.75	97	79.25	88	88.75	87.34	-
Base 2025.09		-	86.13	53.54	55.04	67.87	66.09	71.9	62.92	51	64.31	-
Post-hoc (LlamaGuard) 2025.09		-	97.42	65.56	71.09	73.39	98.03	95.22	79.38	82.57	82.83	-
STAIR-DPO 2025.09		-	96.76	77.61	69.97	92.71	98.2	84.73	77.8	93.05	86.35	-
WJ-SFT 2025.09		-	98.59	73.53	68.99	84.5	96.95	91.1	76.34	91.68	85.21	-
ReSA-SFT (Ours) 2025.09		-	99.56	92.64	82.99	97.61	98.65	94.49	93.46	91.37	93.85	-
Base 2025.09		-	77	61.5	51.75	73.75	43	70.5	47.5	79.5	63.06	-
Post-hoc (LlamaGuard) 2025.09		-	97	75.25	75.5	81.5	96.75	96.5	76	96.25	86.84	-
STAIR-DPO 2025.09		-	96.25	83.75	78.75	95	97	87.25	77.25	99	89.28	-
WJ-SFT 2025.09		-	99.25	81	74.75	89.75	97.5	93.75	73.75	100	88.72	-
ReSA-SFT (Ours) 2025.09		-	99.75	97	93.25	99.25	98.5	98.25	93.5	98.75	97.28	-
Base 2025.09		-	68	14.75	8.5	26.5	33	6.75	11.25	27.25	24.5	-
Post-hoc (LlamaGuard) 2025.09		-	99.75	39.75	65	52	98	86.5	77.25	84.5	75.34	-
STAIR-DPO* 2025.09		-	75.25	43	24	73.75	96.75	28.5	47.25	69	57.19	-
WJ-SFT 2025.09		-	100	34.75	25	69	99.5	37	45.25	73.75	60.53	-
ReSA-SFT (Ours) 2025.09		-	99.5	88	63	94.25	96.25	70.5	88.25	78.75	84.81	-
Base 2025.09		-	80.04	49.57	45.69	49.84	70.19	55.13	44.39	47.9	55.34	-
Post-hoc (LlamaGuard) 2025.09		-	92.91	62.91	77.58	64.06	98.41	95.62	80.29	83.33	81.89	-
STAIR-DPO* 2025.09		-	91.5	75.53	64.62	89.72	97.06	71.9	72.38	91.46	81.77	-
WJ-SFT 2025.09		-	98.77	67.57	67.83	82.94	99.06	80.93	66.85	90.43	81.8	-
ReSA-SFT (Ours) 2025.09		-	99.49	94.27	82.13	96.55	98.99	92.26	92.17	88.8	93.08	-
Base 2025.09		-	71	56.25	39.75	58.75	58.75	52.75	21.5	65.5	53.03	-
Post-hoc (LlamaGuard) 2025.09		-	96	72.5	79.75	76.75	98.5	98.25	76.25	94.5	86.56	-
STAIR-DPO* 2025.09		-	87.75	80.75	70.5	92.25	97.75	68.5	65	98.5	82.63	-
WJ-SFT 2025.09		-	99	73.75	72.5	89.5	100	85.25	58.5	98	84.56	-
ReSA-SFT (Ours) 2025.09		-	99.5	97.25	89.75	98.5	99.25	97.5	93.25	99.25	96.78	-