Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignment

About

Omni-modal Large Language Models (OLLMs) greatly expand LLMs' multimodal capabilities but also introduce cross-modal safety risks. However, a systematic understanding of vulnerabilities in omni-modal interactions remains lacking. To bridge this gap, we establish a modality-semantics decoupling principle and construct the AdvBench-Omni dataset, which reveals a significant vulnerability in OLLMs. Mechanistic analysis uncovers a Mid-layer Dissolution phenomenon driven by refusal vector magnitude shrinkage, alongside the existence of a modal-invariant pure refusal direction. Inspired by these insights, we extract a golden refusal vector using Singular Value Decomposition and propose OmniSteer, which utilizes lightweight adapters to modulate intervention intensity adaptively. Extensive experiments show that our method not only increases the Refusal Success Rate against harmful inputs from 69.9% to 91.2%, but also effectively preserves the general capabilities across all modalities. Our code is available at: https://github.com/zhrli324/omni-safety-research.

Kun Wang, Zherui Li, Zhenhong Zhou, Yitong Zhang, Yan Mi, Kun Yang, Yiming Zhang, Junhao Dong, Zhongxiang Sun, Qiankun Li, Yang Liu• 2026

Related benchmarks

TaskDatasetResultRank
Safety EvaluationHarmBench
Harmbench Score100
76
Safety EvaluationBeaverTails Text
Overall Score99.3
16
Safety EvaluationHBAudio
RSR97.5
12
Safety EvaluationMMSafety Text+Image
RSR99.79
12
Safety EvaluationHolisafe Text+Image
RSR91.13
12
Safety EvaluationVideoSafety Text+Video
RSR100
12
Safety EvaluationOmniSafety (T+I+A)
RSR99.93
12
Safety EvaluationOmniSafety (T+V+A)
RSR99.97
12
Safety EvaluationOmniBench
Accuracy42.47
12
Safety EvaluationBeaverTails Audio 1K
RSR96.66
12
Showing 10 of 18 rows

Other info

Follow for update