Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Skin-R1: Clinical Knowledge-Guided Dermatological Diagnosis Using Vision-Language Models

About

Vision--language models (VLMs) have recently shown promise for assisting clinical reasoning in dermatological diagnosis. However, their trustworthiness and clinical utility remain limited by three key challenges: heterogeneous datasets with inconsistent diagnostic labels and concept annotations, the lack of grounded diagnostic rationales for reliable reasoning supervision, and limited scalability when transferring knowledge from small, densely annotated datasets to large collections with sparse labels. To address these challenges, we propose Skin-R1, a dermatology-oriented VLM that integrates textbook-grounded clinical reasoning supervision with reinforcement learning (RL) to improve the accuracy and robustness of diagnostic prediction. First, we construct a textbook-based reasoning generator that synthesizes hierarchy-aware and differential-diagnosis (DDx) diagnostic trajectories derived from authoritative dermatology knowledge. Second, these trajectories are used for supervised fine-tuning (SFT), establishing a clinically grounded reasoning foundation for the model. Finally, we introduce an RL training framework that incorporates the hierarchical structure of dermatological diseases into the reward design, enabling the model to generalize grounded diagnostic reasoning to large-scale datasets with sparse annotations. Extensive experiments across multiple dermatology benchmarks demonstrate that Skin-R1 consistently improves diagnostic accuracy and robustness compared to state-of-the-art Med-VLM baselines. Ablation studies further highlight the critical role of grounded reasoning supervision introduced during the SFT stage.

Zehao Liu, Weijieying Ren, Jipeng Zhang, Tianxiang Zhao, Jingxi Zhu, Xiaoting Li, Vasant G Honavar• 2025

Related benchmarks

TaskDatasetResultRank
Skin lesion diagnostic classificationBCN20k (In-distribution)
Accuracy53.7
12
Skin lesion diagnostic classificationDerm7pt (In-distribution)
Accuracy70.89
12
Skin lesion diagnostic classificationDermNet In-distribution
Accuracy62.57
12
Skin lesion diagnostic classificationAverage (In-distribution)
Accuracy69.28
12
Skin lesion diagnostic classificationDerm12345 In-distribution
Accuracy89.18
12
Skin lesion diagnostic classificationHAM10k (In-distribution)
Accuracy74.72
12
Skin lesion diagnostic classificationPAD (In-distribution)
Accuracy35.36
12
disease diagnosticDermatology disease diagnostic datasets (In-distribution)
BCN20k Score0.6345
11
disease diagnosticDermatology disease diagnostic datasets (OOD)
ISBI 2016 Score68.28
11
Showing 9 of 9 rows

Other info

Follow for update