Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models

About

Recently, promising progress has been made by open-source vision-language models (VLMs) in bringing their capabilities closer to those of proprietary frontier models. However, most open-source models only publish their final model weights, leaving the critical details of data strategies and implementation largely opaque. In this work, we address VLM post-training from a data-centric perspective, showing the key role of data strategy in developing frontier VLMs. By studying and building our post-training data strategy from scratch, we share detailed insights into the development processes, aiming to benefit the development of competitive models for the open-source community. Our introduced data strategy, together with training recipes and model design, leads to a family of performant VLMs named Eagle2. Specifically, Eagle2-9B achieves state-of-the-art results across various multimodal benchmarks, matching certain competitive models with up to 70B parameters.

Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, Nadine Chang, Karan Sapra, Amala Sanjay Deshmukh, Tuomas Rintamaki, Matthieu Le, Ilia Karmanov, Lukas Voegtle, Philipp Fischer, De-An Huang, Timo Roman, Tong Lu, Jose M. Alvarez, Bryan Catanzaro, Jan Kautz, Andrew Tao, Guilin Liu, Zhiding Yu• 2025

Related benchmarks

TaskDatasetResultRank
Multimodal UnderstandingMMBench
Accuracy74.9
887
Multimodal UnderstandingMM-Vet
MM-Vet Score53.8
664
Visual Question AnsweringChartQA
Accuracy82.3
620
Multimodal UnderstandingMMStar
Accuracy56.4
511
Visual Question AnsweringAI2D
Accuracy79.3
402
Visual Question AnsweringInfoVQA
Accuracy65.8
264
Multimodal UnderstandingMMMU (val)--
211
Visual Question AnsweringDocVQA
Accuracy88
205
Visual Question AnsweringRWQA
Accuracy63.1
79
OCR & Document UnderstandingOCRBench
Score818
77
Showing 10 of 16 rows

Other info

Follow for update