Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions

About

Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro.

Yunheng Li, Hengrui Zhang, Meng-Hao Guo, Wenzhao Gao, Shaoyong Jia, Shaohui Jiao, Qibin Hou, Ming-Ming Cheng• 2026

Related benchmarks

TaskDatasetResultRank
Video CaptioningVDC
Short Accuracy28.8
28
Audiovisual Video CaptioningUGC-VideoCap
Audio Score79.1
26
Audiovisual Video CaptioningSALMONN 2 (test)
Miss Rate20.5
26
Video Captioning EvaluationVidCapBench AE
Overall Accuracy18.2
17
QA performance by Gemini-2.5-Pro based on captionsDaily-Omni (test)
Daily-Omni QA Score61.2
13
QA performance by Gemini-2.5-Pro based on captionsWorld-Sense (test)
World-Sense QA Score34
13
Attribute-level Instruction FollowingAttribute-level Instruction Following Evaluation Set
Acc (1 Attribute)52.3
7
Showing 7 of 7 rows

Other info

Follow for update