Share your thoughts, 1 month free Claude Pro on usSee more

Prefix-based Violation Detection on POLICYGUARDBENCH

93.89Accuracy (N=1)

LLAMA-4-SCOUT-17B-INSTRUCT

Updated 2mo ago

Evaluation Results

Method	Links
LLAMA-4-SCOUT-17B-INSTRUCT 2025.10		93.89	88.54	85.83	83.55	82.37	86.84
LLAMA-3.3-70B-INSTRUCT 2025.10		92.98	84.41	83.68	83.05	81.91	85.21
DEEPSEEK-V3.1 2025.10		92.71	85.87	84.67	83.04	81.29	85.52
GEMMA-3-12B-IT 2025.10		92.27	84.92	83.64	82.58	82.03	85.09
GEMMA-3-27B-IT 2025.10		91.08	87.89	85.26	82.54	80.99	85.55
POLICYGUARD-4B 2025.10		91.01	86.48	84.41	82.76	81.9	85.31
LLAMA-3.2-3B-INSTRUCT 2025.10		90.86	81.99	73.48	63.77	56.93	73.41
CLAUDE-SONNET-4 2025.10		89.94	85.37	84.84	83.09	81.15	84.88
GEMINI-1.5-PRO 2025.10		89.9	87.79	86.67	86.3	85.43	87.22
LLAMA-3.1-8B-INSTRUCT 2025.10		89.76	77.41	67.31	61.21	53.71	69.88
QWEN3-235B-A22B-INSTRUCT 2025.10		89.76	87.52	86.44	85.69	84.98	86.88
QWEN3-4B-INSTRUCT 2025.10		88.32	82.31	80.38	76.88	73.3	80.24
QWEN2.5-72B-INSTRUCT 2025.10		85.08	81.7	83.82	84.04	83.27	83.58
GEMMA-3-4B-IT 2025.10		82.6	64.01	57.81	62.36	63.85	66.13
QWEN3-30B-A3B-INSTRUCT 2025.10		71.99	69.55	72.13	72.73	74.68	72.22
QWEN3-8B 2025.10		61.02	74.29	63.08	55.26	49.11	60.55