Medical Decision-Making on MMLUPH (test)

78.48Accuracy

MAC (Ours w/ GPTs)

Updated 21d ago

Evaluation Results

Method
MAC (Ours w/ GPTs) 2025.07	78.48	78.47	78.72	97.6	76.05
Gemini 2.5 Flash 2025.07	77.34	77.34	77.55	97.47	74.77
GPT-5.1 2025.07	71.35	71.45	72.25	96.79	68.09
Masked Agent Collaboration (MAC) 2025.07	71.35	71.45	72.25	96.79	68.09
Phi4 14B 2025.07	70.46	70.4	71.17	96.69	67.07
Claude Haiku 4.5 2025.07	68.77	68.74	68.99	96.62	65.18
Debate 2025.07	67.93	67.95	69.49	96.4	64.33
Openthinker 32B 2025.07	67.81	67.91	68.49	96.52	64.14
Qwen2.5 32B 2025.07	67.77	67.8	68.08	96.4	64.08
QWQ 32B 2025.07	67.62	67.76	68.37	96.5	63.94
Llama3 instruct 70B 2025.07	66.06	66.02	66.3	96.22	62.18
MoA 2025.07	65.31	65.57	66.98	96.12	61.38
Deepseek-R1 32B 2025.07	62.71	63.29	65.01	95.99	58.64
Qwen2.5 14B 2025.07	62	62	62.4	95.75	57.62
WizardLM 8x22 141B 2025.07	60.68	61.19	64.11	95.61	56.57
SelfMoA 2025.07	60.19	60.86	63.34	95.56	55.99
Mixtral 8x22 141B 2025.07	59.05	59.1	60.31	95.43	54.39
Qwen1.5 Chat 110B 2025.07	45.36	48.73	61.13	93.94	42.06
Med42 8B 2025.07	44.84	45.44	48.82	94.03	38.82
dbrx-instruct 132B 2025.07	44.42	46.14	51.93	93.83	39.11
Medllama2 7B 2025.07	38.96	41.45	53.08	93.4	34.44
HuatuoGPT-o1 8B 2025.07	32.49	32.91	44.03	92.68	27
Qwen1.5 Chat 72B 2025.07	15.29	13.19	48.32	90.64	12.05