Share your thoughts, 1 month free Claude Pro on usSee more

Safety Evaluation on AdvBench (Adversarial Attack Metrics)

100Overall Safety Score

Post-hoc (LlamaGuard)

Updated 1mo ago

Evaluation Results

Method
Post-hoc (LlamaGuard) 2025.09	100	37.5	46.15	57.5	98.65	85.77	60.96	92.31	72.36
STAIR-DPO 2025.09	100	53.27	28.65	94.81	100	69.04	65	97.88	76.08
WJ-SFT 2025.09	100	32.88	27.69	68.65	100	61.15	59.23	99.62	68.65
ReSA-SFT 2025.09	100	87.12	64.23	97.31	100	81.73	88.65	99.62	89.83
STAIR-DPO 2025.09	100	90.38	85.19	99.62	100	92.12	85.19	99.42	93.99
WJ-SFT 2025.09	100	75	75.96	84.62	100	86.54	70.77	99.62	86.56
ReSA-SFT 2025.09	100	97.5	94.23	99.42	100	98.85	95.19	99.62	98.1
Base 2025.09	100	16.35	6.92	31.54	43.65	5.58	14.23	17.88	29.52
Post-hoc (LlamaGuard) 2025.09	100	29.81	51.35	49.23	98.85	87.31	63.08	92.12	71.47
STAIR-DPO* 2025.09	100	54.62	28.65	94.62	84.42	34.04	62.88	94.81	69.25
WJ-SFT 2025.09	100	21.15	20.38	64.23	98.85	34.81	42.88	97.88	60.02
ReSA-SFT 2025.09	100	85.96	64.23	96.73	98.08	74.62	89.04	98.08	88.34
STAIR-DPO* 2025.09	100	90.96	87.69	99.62	96.15	65.58	81.73	99.04	90.1
ReSA-SFT 2025.09	100	96.35	94.04	98.85	100	98.27	94.23	99.42	97.65
STAIR-DPO 2025.09	99.92	74.24	63.83	95.92	99.92	84.41	74.95	99.02	86.53
STAIR-DPO* 2025.09	99.9	75.61	62.43	95.91	95.7	65.36	70.65	98.13	82.96
WJ-SFT 2025.09	99.81	68.08	63.65	81.35	99.04	65.96	59.23	98.65	79.47
ReSA-SFT 2025.09	99.68	88.57	73.38	97.46	99.6	92.58	90.06	97.99	92.42
WJ-SFT 2025.09	99.58	47.96	48.19	70.83	99.19	65.22	53.92	97.27	72.77
WJ-SFT 2025.09	99.57	56.06	56.93	73.29	99.83	86.21	65.89	98.97	79.59
ReSA-SFT 2025.09	99.39	89.85	74.1	97.44	99.65	94.93	89.72	99.1	93.02
Base 2025.09	99.23	63.85	40.38	70.58	48.65	31.73	23.65	67.31	55.67
Post-hoc (LlamaGuard) 2025.09	99.23	75.77	76.54	82.69	96.73	97.12	68.85	98.65	86.95
Post-hoc (LlamaGuard) 2025.09	98.46	76.54	75.38	79.81	97.5	96.73	71.54	97.69	86.71
Post-hoc (LlamaGuard) 2025.09	98.16	52.83	63.75	61.23	97.54	92.72	69.34	87.93	77.94
Base 2025.09	95.85	36.08	30.06	33.52	57.47	34.45	28.58	36.79	44.1
Post-hoc (LlamaGuard) 2025.09	95.85	46.78	64.99	48.6	97.08	94.86	67.03	90.2	75.67
Base 2025.09	95.38	21.35	20.38	46.92	42.69	46.92	39.62	27.31	42.57
Base 2025.09	94.71	39.57	42.52	52.34	49.55	63.59	51.94	38.07	54.04
Base 2025.09	94.23	63.65	54.42	70.96	44.62	67.12	50.96	72.12	64.76