Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning

About

High-quality and carefully curated data is a cornerstone of training medical large language models, as it directly impacts both generalization and robustness to unseen clinical tasks. We investigate strategies for training and data curation to develop a robust multimodal reasoning model in the medical domain. Our work focuses on supervised fine-tuning (SFT) and explores data recipes that leverage structured reasoning traces. Using our proposed data recipe, we scale experiments to a dataset of over 8 million examples and 6.8 billion response tokens, achieving state-of-the-art performance among open-source models across diverse out-of-distribution medical benchmark tasks. Our results further indicate that curating a high-quality, diverse training dataset with varying structured reasoning trace lengths enables the fine-tuned model to self-calibrate its reasoning trajectory lengths based on the downstream task, without explicit supervision. We present key insights, describe the data curation strategy, and outline next steps toward developing robust medical vision-language reasoning system.

Timothy Ossowski, Sheng Zhang, Qianchu Liu, Guanghui Qin, Reuben Tan, Tristan Naumann, Junjie Hu, Hoifung Poon• 2025

Related benchmarks

TaskDatasetResultRank
Medical Visual Question AnsweringSlake
Accuracy84
289
Medical Visual Question AnsweringVQA-RAD
Accuracy79
251
Medical Visual Question AnsweringPathVQA
Accuracy63
103
Image ClassificationBUSI
Accuracy66.03
84
Visual Question AnsweringVQA-RAD
Overall Accuracy74.2
67
Medical Visual Question AnsweringMedXpertQA
Accuracy33.32
52
Visual Question AnsweringPMC-VQA
Accuracy57.14
28
Image ClassificationHAM10000
Accuracy38.14
27
Medical Visual Question AnsweringMedX-M
Accuracy35
18
Medical Visual Question AnsweringPMC
Accuracy55.5
18
Showing 10 of 20 rows

Other info

Follow for update