OpenMedReason: Scientific Reasoning Supervision for Medical Vision-Language Models
About
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We introduce OpenMedReason, a large-scale, open multimodal medical reasoning corpus comprising approximately 450K image-question-answer instances whose reasoning traces are primarily derived from curated biomedical, human-authored scientific articles. OpenMedReason provides high-fidelity supervision beyond synthetic chains of thought, covering diverse medical domain vision modalities such as radiological scans, microscopic images, visible light photographs, charts, and others. We complement it with OpenMedReason-Bench, a held-out benchmark that allows fine-grained evaluation of LVLMs along three complementary axes of capability, including perception, medical knowledge, and rationale, enabling diagnostic evaluation beyond final-answer accuracy. OpenMedReason is a rich training resource that exhibits its effectiveness in both supervised fine-tuning (SFT) and reinforcement-based alignment. Training with OpenMedReason yields a 20% average improvement in VQA accuracy over the base model and achieves performance within 4.2% of the strongest comparable-scale medical LVLMs. Fine-grained performance analysis confirms that the gains are not concentrated in any single axis: OpenMedReason improves perception, medical knowledge, and rationale jointly, and its reasoning traces are preferred over those of the base model in 86.1% of pairwise comparisons. We release the code and dataset at huggingface.co/datasets/neginb/OpenMedReason.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Image Classification | BUSI | Accuracy60.26 | 84 | |
| Visual Question Answering | VQA-RAD | Overall Accuracy72.51 | 67 | |
| Medical Visual Question Answering | MedXpertQA | Accuracy24.95 | 52 | |
| Visual Question Answering | PMC-VQA | Accuracy55.2 | 28 | |
| Image Classification | HAM10000 | Accuracy58.71 | 27 | |
| Image Classification | Brain Tumour MRI | Accuracy75.35 | 16 | |
| Visual Question Answering | PathVQA | Accuracy64.1 | 14 | |
| Image Classification | HyperKvasir | Accuracy71.6 | 8 | |
| Visual Question Answering | OPENMEDREASON-Bench | Accuracy78.51 | 8 | |
| Image Classification | VinDr-Mammo | Accuracy60 | 8 |