Share your thoughts, 1 month free Claude Pro on usSee more

Medical Question Answering on PubMedQA (Accuracy)

81.4Accuracy

HuatuoGPT-o1-70B

Updated 10d ago

Evaluation Results

Method	Links
HuatuoGPT-o1-70B 2025.04		81.4
HuatuoGPT-o1-8B 2025.04		80.1
UltraMedical-70B-3 2025.04		80
o3 2025.07		80
HuatuoGPT-o1-72B 2025.04		79.9
OpenBioLLM-70B 2025.04		79.3
UltraMedical-8B-3.1 2025.04		79.2
OpenBioLLM 70B 2025.07		79
Rewrite 2026.03		78.8
HuatuoGPT-o1-7B 2025.04		78.6
Filter 2026.03		78.5
GPT-4o 2025.07		78.4
Direct 2026.03		78.3
Med42-70B 2025.04		78.1
Base 2026.03		78
Orig. 2026.03		77.8
BioMistral DARE 7B 2025.07		77.7
m1-32B-1K 2025.04		77.6
IDC 2026.03		77.6
MMedS-8B 2025.04		77.5
m1-7B-1K 2025.04		77.5
DeepSeek R1 2025.07		77.2
MedGemma 27B 2025.07		76.8
IQVIA Med-R1 8B 2025.07		76.4
Gemini 2.5 Flash 2025.07		76.2
Med42-8B 2025.04		76
m1-7B-23K 2025.04		75.8
Gemini 2.5 Pro 2025.07		75.8
MEDIC-AD 2026.03		75.6
MedLlama3-8B-v2 2025.04		75.5
Lingshu 2026.03		75.4
Citrus-V 2026.03		74.8
JSL-MedLlama 3 8B v2.0 2025.07		74.2
OpenBioLLM 8B 2025.07		74.1
IDC 2026.03		74
MedGemma 4B 2025.07		73.4
Gemma 3 27B 2025.07		73.4
Filter 2026.03		72.9
Qwen2.5-7B-Instruct 2025.04		72.6
Orig. 2026.03		72.5
Direct 2026.03		72
Qwen2.5-7B-Instruct 2025.04		71.3
Qwen2.5-72B-Instruct 2025.04		71.3
Rewrite 2026.03		71.2
UltraMedical-8B-3 2025.04		71.1
Qwen2.5-72B-Instruct 2025.04		70.8
OpenBioLLM-8B 2025.04		70.1
Base 2026.03		69.5
ReConcile 2025.08		69.3
Qwen2.5-32B-Instruct 2025.04		68.9
TMA-AllCompon 2025.08		68.7
Gemma 3 4B 2025.07		68.4
Qwen2.5-32B-Instruct 2025.04		68
Single-Agent Best 2025.08		66.6
MMed-8B-EnIns 2025.04		63.8
MMed-8B 2025.04		63.4
ORM 2026.03		62
OVM 2026.03		59.4
MCNIG 2026.03		57.7
ReFilter 2026.02		56.8
DyLan 2025.08		56.7
MedAgents 2025.08		56.1
ReFilter 2026.02		56
ReFilter 2026.02		55.2
S-RAG 2026.02		55.2
DyPRAG 2026.02		55.2
DyPRAG 2026.02		54.8
S-RAG 2026.02		54.2
QwenPRM 2026.03		54.1
PRAG 2026.02		54
Vanilla 2026.02		54
PRAG 2026.02		54
MedLlama3-8B-v1 2025.04		52.7
Vanilla 2026.02		51.8
PRAG 2026.02		51.8
S-RAG 2026.02		51.6
DyPRAG 2026.02		50
Vanilla 2026.02		49.8
Granite PRM v2 2026.03		49.8
MathShepherd 2026.03		49.7
IG 2026.03		49
MDAgents 2025.08		48.7
Majority voting 2026.03		46.8
ImplicitPRM 2026.03		44.3
Single sampling 2026.03		36.2
ReFilter 2026.02		35.6
Lingshu 2026.03		35.4
S-RAG 2026.02		35.2
PRAG 2026.02		32
Vanilla 2026.02		30.2
DyPRAG 2026.02		28.4
LLaVA-Med 2026.03		26.4