Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

From Pixels to Words -- Towards Native One-Vision Models at Scale

About

Current vision-language models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment, a modular framework that inevitably fragments pixel-level signals across frames and scatters early pixel-word interactions. In parallel, native VLMs, despite impressive performance on single images, remain largely unexplored in multi-image, video understanding, and spatial intelligence. Hence, we introduce NEO-ov, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end, without any external encoders, auxiliary adapters, or post-hoc fusion. By eliminating module boundaries entirely, NEO-ov enables fine-grained and unified spatiotemporal modeling to emerge natively inside the model. Notably, NEO-ov largely narrows the gap to modular counterparts while excelling at fine-grained visual perception, validating that native "one-vision" architectures are not only feasible but competitive at scale. Beyond empirical performance, we unveil systematic architectural analyses and detailed training recipes to facilitate subsequent native multimodal modeling. Our code and models are publicly available at: https://github.com/EvolvingLMMs-Lab/NEO.

Haiwen Diao, Jiahao Wang, Penghao Wu, Yuhao Dong, Yuwei Niu, Yue Zhu, Zhongang Cai, Weichen Fan, Linjun Dai, Silei Wu, Xuanyu Zheng, Mingxuan Li, Yuanhan Zhang, Bo Li, Hanming Deng, Huchuan Lu, Quan Wang, Lei Yang, Lewei Lu, Dahua Lin, Ziwei Liu• 2026

Related benchmarks

TaskDatasetResultRank
Video UnderstandingMVBench
Accuracy70.7
635
Optical Character RecognitionOCRBench
Score81.6
486
Long Video UnderstandingLVBench--
267
Long Video UnderstandingMLVU
Accuracy69.3
265
Hallucination EvaluationHallusionBench
Accuracy59.8
153
Multi-image ReasoningMuirBench
Accuracy58.2
112
Long Video UnderstandingLongVideoBench
Accuracy63.5
103
Multi-modal Question AnsweringMMMU
Accuracy68.1
98
Video Multimodal UnderstandingVideoMMMU
Accuracy51.6
53
Multi-Image Visual ReasoningBLINK
Accuracy62.8
51
Showing 10 of 11 rows

Other info

Follow for update