Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration

About

High-resolution Multimodal Large Language Models (MLLMs) face prohibitive computational costs during inference due to the explosion of visual tokens. Existing acceleration strategies, such as token pruning or layer sparsity, suffer from severe "backbone dependency", performing well on Vicuna or Mistral architectures (e.g., LLaVA) but causing significant performance degradation when transferred to architectures like Qwen. To address this, we leverage truncated matrix entropy to uncover a universal three-stage inference lifecycle, decoupling visual redundancy into universal Intrinsic Visual Redundancy (IVR) and architecture-dependent Secondary Saturation Redundancy (SSR). Guided by this insight, we propose HalfV, a framework that first mitigates IVR via a unified pruning strategy and then adaptively handles SSR based on its specific manifestation. Experiments demonstrate that HalfV achieves superior efficiency-performance trade-offs across diverse backbones. Notably, on Qwen25-VL, it retains 96.8\% performance at a 4.1$\times$ FLOPs speedup, significantly outperforming state-of-the-art baselines. Our code is available at https://github.com/civilizwa/HalfV.

Jiaqi Shi, Yuechan Li, Xulong Zhang, Xiaoyang Qu, Jianzong Wang• 2026

Related benchmarks

TaskDatasetResultRank
Object Hallucination EvaluationPOPE
Accuracy83.1
2019
Text-based Visual Question AnsweringTextVQA
Accuracy76.4
962
Science Question AnsweringScienceQA
Accuracy87.5
791
Diagram Question AnsweringAI2D
AI2D Accuracy81.4
387
Multi-modal EvaluationMME
MME Score2.12e+3
160
Multimodal BenchmarkingMMBench
Accuracy80.9
90
Video UnderstandingMSRVTT
Acc56.9
43
Video UnderstandingTGIF
Accuracy46.5
42
Video UnderstandingMSVD
Accuracy70.8
39
Multimodal UnderstandingMLLM Evaluation Suite GQA MME POPE SQA VQAtext VizWiz MMBen AI2D v1.5 v1.6 Qwen2.5 (test)
GQA Score62.5
12
Showing 10 of 10 rows

Other info

Follow for update