Red Teaming

Benchmarks

Dataset Name	SOTA Method	Metric
50 harmful goals (Manual evaluation)	PAIR	Hard ASR100	30	1mo ago
CatQA	SafeTransformer	ASR0	20	1mo ago
AdversarialQA	SafeTransformer	ASR0	20	1mo ago
Religious Discrimination principle v1 (test)	QCI	Mean Best Category Score5.32	12	1mo ago
Illegal Activity principle v1 (test)		Mean Score (Best Category)-2.73	12	1mo ago
AI Supremacy principle v1 (test)	CRL	Mean Best Category Score11.7	12	1mo ago
AdvBench (test)	AMIS	ASR88	8	15d ago
DailyDialog against DialoGPT-large	BRT (e+r)	RSR40	8	1mo ago
DailyDialog against BB-3B	BRT (e+r)	RSR40.2	8	1mo ago
ConvAI2 (filtered hard positive)	BRT (e+r)	RSR2,120	7	1mo ago
Bloom ZS (filtered hard positive)	BRT (e+r)	RSR15.6	7	1mo ago
BAD Against Friend Chat (test)	BRT (e)	RSR64.2	7	1mo ago
BAD Against Marv (test)	BRT (s+r)	RSR88.1	7	1mo ago
Korean red teaming dataset (test)	Exaone-3.5-2.4B-inst	Attack Success Rate0.5797	5	1mo ago
HarmBench Claude-Sonnet-3.5 (held-out test)	AGENTICRED	ASR60	5	1mo ago
HarmBench Llama-3-8B (test)	AGENTICRED	ASR0.98	5	1mo ago
HarmBench Llama-2-7B (test)	AutoDAN-Turbo	ASR36	5	1mo ago
KT RAIC proprietary Korean red-teaming dataset		Attack Success Rate54	4	29d ago
Wan Seed-free generation 2.2	ART	Violence Rate40	3	1mo ago
Hunyuan-Video Seed-free generation	TEAR	Violence Rate60	3	1mo ago
HarmBench (test)	DIALTREE	ASR@185.1	3	1mo ago
HarmBench Claude-4-Sonnet (test)	DIALTREE	ASR@171	3	1mo ago
HarmBench gpt-4o-2024-08-06 (test)	AdvReasoning	ASR86	3	1mo ago
HarmBench gpt-3.5-turbo-0125 (test)	TransferAttack	ASR80	3	1mo ago
ConvAI2 (test)	BRT (e)	P Score186	3	1mo ago

Showing 25 of 25 rows