Share your thoughts, 1 month free Claude Pro on usSee more

Medical Reasoning on RareDis Sub (test)

80Symptoms Accuracy

Llama-3.1-8B-Instruct + MedSSR

Updated 4d ago

Evaluation Results

Method	Links
Llama-3.1-8B-Instruct + MedSSR 2026.04		80	80.3	64.96	71.99	82.89	59.79	86.38	75.19
Qwen3-8B-Base + MedSSR 2026.04		79.71	80.89	68.19	72.22	76.64	57.51	78.62	73.4
Llama-3.1-8B-Instruct + Fully-Supervised 2026.04		76.27	77.72	60.18	67.13	81.09	54.55	83.45	71.48
HuatuoGPT-o1-8B 2026.04		74.9	78.47	61.64	70.1	79.53	52.89	78.68	70.89
m1-7B-23K 2026.04		74.51	74.64	57.82	69.44	68.59	52.37	72.76	67.16
UltraMedical-3.1-8B 2026.04		73.53	73.01	57.28	65.28	77.47	50.83	73.79	67.31
MedReason-8B 2026.04		73.53	74.39	62.85	70.43	76.43	57.1	75.33	70.01
Qwen3-8B-Base + Fully-Supervised 2026.04		73.24	75.41	60.65	65.97	70.39	54.41	72.24	67.47
MedPRM-8B 2026.04		73.14	76	61.79	66.2	78.45	52.37	78.97	69.56
Llama-3.1-8B-Instruct 2026.04		71.67	74.28	60.85	64.81	79.28	52.97	76.55	68.63
Qwen3-8B 2026.04		68.63	72.01	57.95	66.44	70.39	52.18	68.28	65.13
Qwen3-8B-Base 2026.04		67.16	70.74	54.92	68.75	59.05	49.3	67.41	62.48
R1-Distill-Llama-8B 2026.04		53.58	63.04	45.49	60.19	55.52	46.57	63.28	55.38