Jailbreak Detection

Benchmarks

Dataset Name	SOTA Method	Metric
AdvBench Llama2-7B	GradSafe	AUROC97.9	88	1mo ago
Average of six attacks	GradSafe	Avg Success Rate0	38	3mo ago
Jailbreak data (70/30 stratified)	LLMScan	AUC100	32	2mo ago
Zulu	IMAG	Accuracy92	30	4mo ago
Base64	IMAG	Accuracy100	30	4mo ago
DrAttack	GradSafe	Accuracy99	30	4mo ago
PAIR	IMAG	Accuracy98	30	4mo ago
AutoDAN	IMAG	Accuracy99	30	4mo ago
GCG	IMAG	Accuracy99	30	4mo ago
GoalFrameBench	FrameShield-Crit	Accuracy94	24	4mo ago
MM-SafetyBench	Mahal-OOD	AUROC99.18	23	1mo ago
AdvBench Vicuna-7B	MTK	AUROC95.7	16	1mo ago
AdvBench Mistral-7B	MTK	AUROC99	16	1mo ago
AdvBench Llama3-8B	GradSafe	AUROC0.967	16	1mo ago
GoalFrameBench (seed prompts)	FrameShield-Last	Accuracy97	16	4mo ago
JailbreaksOverTime (test)	MLJailDe	Throughput (items/s)38.06	15	1mo ago
GCG attacks, Databricks Dolly 15K, and OR-Bench PMPs (test)	SelfDefend	True Positive Rate (TPR)99	15	1mo ago
DrAttack	SelfDefend (Intent)	ASR3	15	3mo ago
AutoDAN	GradientCuff	Attack Success Rate (ASR)0	15	3mo ago
GCG		ASR13	15	3mo ago
ChatGPT Jailbreak Prompts		Recall100	15	4mo ago
Wildjailbreak	Apriel Guard	F1 Score96	15	4mo ago
Jailbreak V28K	MTK	AUROC96.4	14	1mo ago
OKT		Correlation Score1	13	4mo ago
SB	gpt-4o-mini	COR98.33	13	4mo ago

Showing 25 of 94 rows