JailbreakBench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Jailbreak Attack | JailbreakBench | ASR@100 | 132 | |
| Jailbreak Attack | JailbreakBench | ASR96.33 | 54 | |
| Jailbreak Attack | JailbreakBench (JBB) | ASR0 | 54 | |
| Adversarial and Jailbreaking Attack Detection | JailbreakBench | AUROC0.8622 | 20 | |
| Red-teaming Attack Success Rate | JailbreakBench (test) | ASR (Vicuna)82 | 18 | |
| Safety Evaluation | JailbreakBench | Reject Rate99 | 13 | |
| Jailbreak Defense | JailbreakBench Qwen-2.5-7B | ASR0 | 12 | |
| Jailbreak Defense | JailbreakBench (JBB) | FNR36.11 | 12 | |
| Jailbreak Detection | JailBreakBench Single Turn 35 | F1 Score98 | 10 | |
| Jailbreak Attack | JailbreakBench | Safe2Harm ASR (Answer)97.98 | 10 | |
| Jailbreak Attack | JailbreakBench GCG | ASR0.58 | 10 | |
| Jailbreak Attack | JailbreakBench PAIR | Attack Success Rate (ASR)82 | 10 | |
| Jailbreak Attack | JailbreakBench (test) | ASR (Refusal)32 | 10 | |
| Jailbreak Defense | JailbreakBench Vicuna-13B | ASR0 | 7 | |
| Jailbreak Defense | JailbreakBench LLaMA-3-8B | ASR0 | 7 | |
| Jailbreaking | JailbreakBench | Jailbroken Behaviors (k)1 | 5 | |
| Jailbreak Defense Efficiency and Effectiveness | JailbreakBench Summary Aggregates | Avg ASR1 | 5 | |
| Jailbreak Attack | JailbreakBench 2024a | Average Latency (s)2.4 | 4 |
Showing 18 of 18 rows