Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions
About
Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Audiovisual Video Captioning | SALMONN 2 (test) | Miss Rate20.5 | 37 | |
| Video Captioning | VDC | Short Accuracy28.8 | 35 | |
| Audiovisual Video Captioning | UGC-VideoCap | Audio Score79.1 | 34 | |
| Audiovisual Understanding & Reasoning | Daily-Omni | Score58.56 | 33 | |
| Video Question Answering | WorldSense | Accuracy34 | 24 | |
| Video Question Answering | DailyOmni | Accuracy61.2 | 19 | |
| Video Captioning Evaluation | VidCapBench AE | Overall Accuracy18.2 | 17 | |
| Instruction Following | OmniCap-IF (test) | Overall CSR24.52 | 17 | |
| Audiovisual Understanding & Reasoning | World-Sense | Score32.03 | 14 | |
| QA performance by Gemini-2.5-Pro based on captions | Daily-Omni (test) | Daily-Omni QA Score61.2 | 13 |