Share your thoughts, 1 month free Claude Pro on usSee more

Safety Evaluation on AdvBench (Adversarial Attack Metrics)

100Overall Safety Score

Post-hoc (LlamaGuard)

Updated 27d ago

Evaluation Results

Method
Post-hoc (LlamaGuard) 2025.09	100	37.5	46.15	57.5	98.65	85.77	60.96	92.31	72.36	-	-
STAIR-DPO 2025.09	100	53.27	28.65	94.81	100	69.04	65	97.88	76.08	-	-
WJ-SFT 2025.09	100	32.88	27.69	68.65	100	61.15	59.23	99.62	68.65	-	-
ReSA-SFT 2025.09	100	87.12	64.23	97.31	100	81.73	88.65	99.62	89.83	-	-
STAIR-DPO 2025.09	100	90.38	85.19	99.62	100	92.12	85.19	99.42	93.99	-	-
WJ-SFT 2025.09	100	75	75.96	84.62	100	86.54	70.77	99.62	86.56	-	-
ReSA-SFT 2025.09	100	97.5	94.23	99.42	100	98.85	95.19	99.62	98.1	-	-
Base 2025.09	100	16.35	6.92	31.54	43.65	5.58	14.23	17.88	29.52	-	-
Post-hoc (LlamaGuard) 2025.09	100	29.81	51.35	49.23	98.85	87.31	63.08	92.12	71.47	-	-
STAIR-DPO* 2025.09	100	54.62	28.65	94.62	84.42	34.04	62.88	94.81	69.25	-	-
WJ-SFT 2025.09	100	21.15	20.38	64.23	98.85	34.81	42.88	97.88	60.02	-	-
ReSA-SFT 2025.09	100	85.96	64.23	96.73	98.08	74.62	89.04	98.08	88.34	-	-
STAIR-DPO* 2025.09	100	90.96	87.69	99.62	96.15	65.58	81.73	99.04	90.1	-	-
ReSA-SFT 2025.09	100	96.35	94.04	98.85	100	98.27	94.23	99.42	97.65	-	-
STAIR-DPO 2025.09	99.92	74.24	63.83	95.92	99.92	84.41	74.95	99.02	86.53	-	-
STAIR-DPO* 2025.09	99.9	75.61	62.43	95.91	95.7	65.36	70.65	98.13	82.96	-	-
WJ-SFT 2025.09	99.81	68.08	63.65	81.35	99.04	65.96	59.23	98.65	79.47	-	-
ReSA-SFT 2025.09	99.68	88.57	73.38	97.46	99.6	92.58	90.06	97.99	92.42	-	-
WJ-SFT 2025.09	99.58	47.96	48.19	70.83	99.19	65.22	53.92	97.27	72.77	-	-
WJ-SFT 2025.09	99.57	56.06	56.93	73.29	99.83	86.21	65.89	98.97	79.59	-	-
ReSA-SFT 2025.09	99.39	89.85	74.1	97.44	99.65	94.93	89.72	99.1	93.02	-	-
Base 2025.09	99.23	63.85	40.38	70.58	48.65	31.73	23.65	67.31	55.67	-	-
Post-hoc (LlamaGuard) 2025.09	99.23	75.77	76.54	82.69	96.73	97.12	68.85	98.65	86.95	-	-
Post-hoc (LlamaGuard) 2025.09	98.46	76.54	75.38	79.81	97.5	96.73	71.54	97.69	86.71	-	-
Post-hoc (LlamaGuard) 2025.09	98.16	52.83	63.75	61.23	97.54	92.72	69.34	87.93	77.94	-	-
Base 2025.09	95.85	36.08	30.06	33.52	57.47	34.45	28.58	36.79	44.1	-	-
Post-hoc (LlamaGuard) 2025.09	95.85	46.78	64.99	48.6	97.08	94.86	67.03	90.2	75.67	-	-
Base 2025.09	95.38	21.35	20.38	46.92	42.69	46.92	39.62	27.31	42.57	-	-
Base 2025.09	94.71	39.57	42.52	52.34	49.55	63.59	51.94	38.07	54.04	-	-
Base 2025.09	94.23	63.65	54.42	70.96	44.62	67.12	50.96	72.12	64.76	-	-
No Defense 2025.08	-	-	-	-	-	-	-	-	-	40	32
PPL 2025.08	-	-	-	-	-	-	-	-	-	40	32
Llama Guard 3 2025.08	-	-	-	-	-	-	-	-	-	0	10
GuardReasoner 2025.08	-	-	-	-	-	-	-	-	-	2	6
ReasoningGuard 2025.08	-	-	-	-	-	-	-	-	-	2	2
ReasoningGuard + LG 2025.08	-	-	-	-	-	-	-	-	-	0	0
ReasoningGuard + GR 2025.08	-	-	-	-	-	-	-	-	-	0	0
No Defense 2025.08	-	-	-	-	-	-	-	-	-	26	22
PPL 2025.08	-	-	-	-	-	-	-	-	-	26	22
Llama Guard 3 2025.08	-	-	-	-	-	-	-	-	-	2	2
GuardReasoner 2025.08	-	-	-	-	-	-	-	-	-	0	0
ReasoningGuard 2025.08	-	-	-	-	-	-	-	-	-	0	0
ReasoningGuard + LG 2025.08	-	-	-	-	-	-	-	-	-	0	0
ReasoningGuard + GR 2025.08	-	-	-	-	-	-	-	-	-	0	0