Opir: Efficient Multi-Task Safety Classification for Toxicity, Jailbreaks, Hate Speech, and Harmful Content
About
Real-time safety filtering for large language model (LLM) applications requires classifiers that can detect unsafe prompts, toxic language, jailbreak attempts, and unsafe responses without the cost profile of large guardrail models, and that can distinguish benign sensitive text from genuinely covert harmful content. In this paper, we introduce Opir, a family of encoder-based guardrail models built on the GLiClass architecture. Opir includes multi-task models for binary safe/unsafe classification, multi-label toxicity classification, jailbreak classification, and zero-shot unsafe prompt and response categorization. We also release edge variants with fewer than 100M parameters dedicated to binary safe/unsafe categorization. The models are trained on a three-level taxonomy containing 996 categories across 16 top-level labels, 126 mid-level labels, and 854 leaf labels. Opir's training data combines taxonomy-grounded unsafe prompts, adversarially mined hard negatives, benign safety-preserving examples, generated response examples, multilingual translations, and portions of the Aegis2 and WildGuard training subsets. We also open-sourced an evaluation harness that supports GLiClass and GLiNER2 backends as well as decoder-based models, and covers binary safety classification, multi-label categorization, toxicity, jailbreak detection, prompt safety, response safety, response refusal, and prompt subcategory views across public benchmark families. Across an expanded comparison spanning 12 safety-classification tasks and 17 category tasks against eight contemporary guardrail systems -- including both GLiNER2-based and generative guardrail models -- Opir variants are competitive on or ahead of the strongest open-weight baselines on the majority of benchmark datasets while operating with a substantially smaller deployment footprint.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Safety Classification | AegisSafetyTest V2 | -- | 14 | |
| Binary Safety Classification | wildguard prompt safety | Macro F197.91 | 11 | |
| Inference Efficiency | 1024-token sequences (inference summary) | Throughput (Samples/s)499.5 | 11 | |
| Binary Safety Classification | ToxicChat jailbreaking | Macro F166.34 | 11 | |
| Binary Safety Classification | oai_safety OpenAI moderation | Macro F163.97 | 11 | |
| Multi-label Safety Categorization | or_bench 80k | Macro Accuracy50.32 | 8 | |
| Multi-label Safety Categorization | OpenAI Moderation | Macro Accuracy47.67 | 4 | |
| Multi-label Safety Categorization | aegis categories | Macro Accuracy62.84 | 4 | |
| Multi-label Safety Categorization | simplest | Macro Accuracy86.68 | 4 | |
| Multi-label Safety Categorization | SimpleSafetyTests | Macro Accuracy91.38 | 4 |