Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning

About

Vision-language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the safety surface: risks can arise from multimodal question answering, assistant responses, and cross-modal composition, while moderation policies may vary across products, regions, and deployment stages. Most existing guardrails either rely on fixed taxonomies or target only a narrow set of interaction settings, which limits their adaptability when safety rules change at deployment time. We present \textbf{SingGuard}, a policy-adaptive multimodal guardrail model family for safety assessment in multimodal conversations. SingGuard treats the active policy as a runtime input: given natural-language rules, it checks the target content against the active policy rule by rule and predicts both the safety label and the triggered rule. To balance efficiency and interpretability, SingGuard supports fast, hybrid, and slow inference regimes along a fast-to-slow reasoning spectrum, ranging from direct safety judgments to policy-grounded deliberation. We further optimize this behavior with fast--slow decoupled reinforcement learning. We also introduce \textbf{SingGuard-Bench}, a multimodal guardrail benchmark with 56{,}340 examples spanning 80+ fine-grained risk types across multimodal QA, adversarial attack, and dynamic-rule evaluation settings, including cross-modal joint-risk cases where each modality is harmless in isolation but their composition implies unsafe intent. Across six benchmark families (35 datasets), SingGuard achieves state-of-the-art average F1 in every family. Dynamic-rule evaluation further shows improved policy-following accuracy from 0.6465 to 0.7415 under runtime policy shifts. Our code is available at https://github.com/inclusionAI/Sing-Guard.

SingGuard Team• 2026

Related benchmarks

TaskDatasetResultRank
Text-based safety moderationBeavertails
F1 Score86.01
60
Harmfulness DetectionOpenAI Moderation
Macro F1 Score75.61
59
Safety EvaluationUnsafeBench
F1 Score80.02
39
Safety ClassificationWildGuardMix
F1 Score89.54
29
Text Query SafetyHarmBench
F1 Score94.85
28
Text Query SafetyEXPGUARD (test)
F1 Score93.93
28
Text Query SafetyXGuard (test)
F1 Score91.17
28
Text Query SafetyAegis2
F1 Score84.95
28
Text-based safety moderationWildGuard
F1 Score80.26
26
Multilingual text query safetyPolyGuardPrompts
F1 Score88.02
19
Showing 10 of 34 rows

Other info

GitHub

Follow for update