Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training

About

Clinical decision-making routinely demands reasoning over heterogeneous data, yet existing multimodal language models (MLLMs) remain largely vision-centric and fail to generalize across clinical specialties. To bridge this gap, we introduce QoQ-Med-7B/32B, the first open generalist clinical foundation model that jointly reasons across medical images, time-series signals, and text reports. QoQ-Med is trained with Domain-aware Relative Policy Optimization (DRPO), a novel reinforcement-learning objective that hierarchically scales normalized rewards according to domain rarity and modality difficulty, mitigating performance imbalance caused by skewed clinical data distributions. Trained on 2.61 million instruction tuning pairs spanning 9 clinical domains, we show that DRPO training boosts diagnostic performance by 43% in macro-F1 on average across all visual domains as compared to other critic-free training methods like GRPO. Furthermore, with QoQ-Med trained on intensive segmentation data, it is able to highlight salient regions related to the diagnosis, with an IoU 10x higher than open models while reaching the performance of OpenAI o4-mini. To foster reproducibility and downstream research, we release (i) the full model weights, (ii) the modular training pipeline, and (iii) all intermediate reasoning traces at https://github.com/DDVD233/QoQ_Med.

Wei Dai, Peilin Chen, Chanakya Ekbote, Paul Pu Liang• 2025

Related benchmarks

TaskDatasetResultRank
Medical Visual Question AnsweringSlake
Accuracy68.5
247
Medical Visual Question AnsweringVQA-RAD
Accuracy74
228
ECG ClassificationPTBXL Super
Macro AUC69.29
136
Medical Visual Question AnsweringPathVQA
Accuracy63.5
80
ECG ClassificationCSN
Macro AUC85.41
51
ECG ClassificationPTB-XL
AUROC83.76
26
ECG ClassificationCPSC
AUC0.8898
24
Medical Visual Question AnsweringMedX-M
Accuracy25
18
Medical Visual Question AnsweringPMC
Accuracy51
18
ECG ClassificationPTB-XL Form
AUC81.01
17
Showing 10 of 36 rows

Other info

Follow for update