HarmBench

Benchmarks

Task Name	Dataset Name	SOTA Result
Jailbreak Attack	HarmBench	Attack Success Rate (ASR)100	624
Jailbreak Attack	HarmBench (test)	ASRHB100	276
Red-Teaming	HarmBench	ASR96.3	244
Safety Evaluation	HarmBench	ASR0	153
Safety Evaluation	Harmbench	Harmbench Score0.06	127
Jailbreaking	HarmBench	Attack Success Rate (ASR)84	110
Response Harmfulness Detection	HarmBench	F1 Score98.94	100
Jailbreak Defense	HarmBench	PAIR ASR0	91
Safety Alignment	HarmBench	ASR0	88
Unsafe Robustness	HarmBench	Unsafe Rate1	72
Jailbreak Robustness	HarmBench	HarmBench ASR0	72
Multimodal Jailbreak Attack	HarmBench	ASR0	62
Safety Alignment Breaking Prevention	HarmBench	Harmful Score (%)0	60
Jailbreak Attack Success Rate	HarmBench	Attack Success Rate (Generated)97	55
Jailbreaking	HARMBENCH 159 standard behaviors (test)	ASR0	55
Harmful Prompt Refusal	HarmBench	ASR0	52
Jailbreak	HarmBench	Toxicity Score1.01	50
Jailbreak	HarmBench Standard Behaviours (200 examples)	ASR0	48
Jailbreak Attack	HarmBench-191 (dev)	Attack Success Rate (ASR)97.4	42
Single-turn Jailbreak Attack	HarmBench single-turn	ASR91	41
Refusal Ablation and Jailbreak Attack Success	HARMBENCH	Attack Success Rate (ASR)96.27	40
Controllability	HarmBench	HarmBench Score87.5	40
Safety Evaluation	Harmbench	ASR3.5	39
Safety Evaluation	HarmBench	PAIR93.25	39
Malicious Prompt Refusal	HarmBench	Refusal Rate96	38

Showing 25 of 180 rows

...