SOTA Multilingual Multiple-Choice Question Answering on HeadQA 1.0 (test) and PapersWithCode

Method
o4-mini 2026.01	88.88	90.28	90.22	89.97	89.48	87.08	90.77	90.5	86.74	47.23	76.07	82.47	84.14
Gemini-2.5-Flash 2026.01	88.83	88.11	90.35	89.6	88.84	88.96	90.27	90.61	88.23	78.26	80.43	83.53	87.17
GPT-4.1 2026.01	88.57	90.05	90.19	89.59	89.51	86.14	90.52	89.28	86.67	58.72	67.49	78.58	83.78
GPT-5 2026.01	88.3	90.44	89.99	89.61	89.11	86.01	90.41	90.86	86.71	57.47	71.58	79.55	84.17
DeepSeek-R1 2026.01	88.28	89.64	89.63	89.25	87.85	86.69	90.1	89.85	84.13	65.64	69.79	77.36	84.02
Claude-4.0-Sonnet 2026.01	87.78	89.56	89.97	89.32	88.42	87.83	90.32	90.72	80.47	55.56	70.72	76.12	83.07
LLaMA-4-Maverick 2026.01	87.73	89.55	90.02	89.57	87.02	86	89.98	90.05	84.15	54.47	69.27	77.79	82.97
DeepSeek-V3 2026.01	87.64	89.67	88.59	88.74	87.44	84.1	88.48	89.02	77.07	51.05	58.45	64.69	79.58
GPT-4o 2026.01	86.96	87.87	88.95	88.29	87.69	85.07	88.81	89.18	83.97	37.49	61.44	73.68	79.95
gpt-oss-120B 2026.01	86.91	88	88.48	88.13	86.81	85.46	88.13	88.26	77.86	52.62	68.26	76.32	81.27
GPT-4.1-mini 2026.01	86.13	88.59	87.75	87.61	86.53	83.05	88.89	88.41	77.55	27.2	61.77	72.59	78.01
GPT-5-mini 2026.01	85.53	88.36	87.9	87.37	86.53	82.71	88.22	88.34	77.87	33.48	59.4	73.02	78.23
QwQ-32B 2026.01	85.03	86.24	86.57	85.38	82.29	79.6	86.68	86	62.21	23.75	29.49	29.87	68.59
Baichuan-M2-32B 2026.01	84.76	86.1	83.98	82.91	81.52	78.42	83.88	83.72	34.33	25.98	33.35	29.88	65.74
HuatuoGPT-o1-72B 2026.01	84.34	85.54	86.2	84.46	83.03	78.55	86.84	86.85	62.18	44.44	44.91	29.51	71.41
LLaMA-4-Scout 2026.01	84.12	87.1	86.31	86.83	84.23	82.29	86.43	87	77.98	50.25	50.64	69.34	77.71
Qwen3-14B-thinking 2026.01	83.59	86.03	84.71	84.81	83.42	80.83	84.66	85.02	64.36	17.63	24.89	27.53	67.29
Qwen2.5-72B 2026.01	83.21	85.62	84.48	84.17	81.56	78.96	85.1	84.97	54.45	43.9	41.74	43.98	71.01
gpt-oss-20B 2026.01	82.64	78.47	84.91	84.63	82.95	81.27	85.03	84.74	68.96	33.72	61.98	69.13	74.87
MedGemma-27B 2026.01	82.2	86.11	84.85	84.64	81.91	80.72	85.48	85.96	77.75	18.04	50.94	65.88	73.71
Qwen3-8B-thinking 2026.01	81.15	83.88	82.17	82.52	79.96	78.42	81.9	82.18	36.87	14.73	17.74	12.06	61.13
Qwen3-14B 2026.01	80.88	83.93	80.96	80.53	76.98	73.88	80.59	80.92	46.86	15.01	24.17	27.17	62.66
Phi-4-Reasoning 2026.01	79.89	85.5	84.47	84.67	83.64	79.93	78.34	85.09	71.75	17.47	48.91	37.77	69.79
DeepSeek-R1-Qwen3-8B 2026.01	79.34	82.46	77.59	77.49	75.66	73.91	78.48	78.63	36.65	11.08	10.83	13.17	57.94
LLaMA-3.1-70B 2026.01	78.36	85.67	84.26	83.43	77.57	67.09	83.76	84.28	70.4	41.28	45.09	49.39	70.88
Qwen2.5-14B 2026.01	77.17	80.3	76.86	75.79	72.86	68.43	75.99	75.73	37.04	26.76	35.73	35.38	61.5
Claude-3.5-Haiku 2026.01	76.96	83.14	82.79	81.82	79.36	74.99	83.34	83.04	59.26	19.45	42.53	42.95	67.47
Qwen3-8B 2026.01	76.95	80.75	75.54	76.37	72.29	67.65	75.36	75.22	31.28	10.53	22.73	11.42	56.34
GPT-4o-mini 2026.01	76.93	83.54	82.68	81.97	78.1	72.34	82.37	82.18	69.38	35.69	48.67	58.1	71
Gemma-3-27B 2026.01	76.73	82.14	80.95	80.88	77.86	76.83	80.71	81.34	71.33	23.93	52.41	62.27	70.62
Qwen3-4B-thinking 2026.01	76.64	80.4	78.85	78.31	75.53	72.08	77.34	78.18	29.54	17.14	13.21	8.99	57.18
GPT-4.1-nano 2026.01	76.59	84.59	81.86	80.96	74.56	72.63	82.2	81.2	59.13	33.51	44.72	53.54	68.79
Mistral-Small-3.1-24B 2026.01	76.22	83.46	80.23	80.87	74.67	68.63	81.02	81.13	38.85	11.82	13.73	20.57	59.27
HuatuoGPT-o1-7B 2026.01	73.8	69.96	72.11	71.8	65.12	62.83	72.32	70.88	7.32	7.04	7.19	6.26	48.89
Qwen3-4B 2026.01	72.91	76.47	70.4	70.43	66.57	61.1	70	69.67	19.33	14.16	16.25	9.66	51.41
HuatuoGPT-o1-70B 2026.01	72.87	85.19	84.22	84.27	72.69	77.43	84.77	84.83	74.9	43.44	50.15	56.66	72.62
Qwen2.5-7B 2026.01	71.29	74.45	68.09	65.88	63.1	57.78	66.76	66.51	32.25	9.13	24.63	18.55	51.54
Gemma-3-12B 2026.01	70.8	78.31	75.85	75.65	71.93	69.44	75.22	75.47	62.87	11.15	43.19	54.35	63.69
Phi-4 2026.01	70.17	84.6	81.31	81.12	72.99	66.18	80.93	80.43	52.61	23.43	38.22	34.67	63.89
GPT-5-nano 2026.01	67.63	80.66	78.25	77.47	74.14	66.74	75.31	77.05	57.85	17.02	41.53	50.74	63.7
HuatuoGPT-o1-8B 2026.01	64.48	73.33	70.71	70.5	62.24	54.89	69.3	69.15	50.34	9.62	9.04	8.46	51.01
LLaMA-3.3-70B 2026.01	61.85	86.28	85.89	84.91	65.94	76.32	85.67	85.93	72.72	41.81	47.93	51.22	70.54
Qwen2.5-3B 2026.01	61.05	66.95	58.48	55.81	50.25	44.89	56.09	55.37	16.09	9.17	13.37	11.75	41.61
Qwen3-1.7B 2026.01	57.75	65.09	55.49	52.98	45.46	42.18	52.99	52.82	23.75	22.52	26.3	23.17	43.38
LLaMA-3.1-8B 2026.01	56.79	74.27	63.01	62.34	50.87	42.49	62.7	62.16	37.13	23.72	19.75	14.36	47.47
MedGemma-4B 2026.01	55	69.39	63.51	61.79	55.02	50.91	62.94	62.57	45.48	20.66	21.98	30.95	50.02
Gemma-3-4B 2026.01	52.83	64.58	59.58	58.23	53.69	48.51	59.74	59.42	42.89	12.9	25.55	20.01	46.49
Bio-Medical-LLaMA-3-8B 2026.01	50.07	69.06	59.37	58.85	49.4	44.72	57.13	54.77	38.93	35.79	34.01	33.82	48.83
LLaMA-3.2-3B 2026.01	49.2	64.07	53.22	49.62	41.37	31.63	50.68	43.97	33.28	14.19	18.89	17.28	38.95
MedReason-8B 2026.01	47.48	63.47	18.4	14.44	48.26	41.14	26.6	34.88	12.92	8.11	13.19	11.66	28.38
Phi-4-mini 2026.01	42.01	71.17	52.71	55.27	39.4	31.97	50.59	50.39	30.29	14.39	19.51	17.02	39.56
MediPhi 2026.01	38.13	68.04	62.01	61.21	31.25	32.76	57.91	53.8	16.53	14.77	15.83	17.52	39.15
Phi-4-mini-Reasoning 2026.01	36.42	74.05	52.82	60.22	26.83	14.56	49.83	44.06	28.71	20.84	18.5	19.8	37.22
OpenBioLLM-70B 2026.01	31.62	73.19	74.86	52.14	32.53	31.09	58.81	70.78	31.04	10.13	11.79	23.73	41.81
Mistral-7B-v0.3 2026.01	28.39	36.61	21.04	40.89	24.09	27.67	30.07	37.35	18.13	14.69	10.4	9.78	24.93
OpenBioLLM-8B 2026.01	22.73	35.28	24.55	32.01	8.99	12.06	34.79	35.48	11.49	6.95	10.1	11.1	20.46

o4-mini

2026.01

88.88

90.28

90.22

89.97

89.48

87.08

90.77

90.5

86.74

47.23

76.07

82.47

84.14

Gemini-2.5-Flash

2026.01

88.83

88.11

90.35

89.6

88.84

88.96

90.27

90.61

88.23

78.26

80.43

83.53

87.17

GPT-4.1

2026.01

88.57

90.05

90.19

89.59

89.51

86.14

90.52

89.28

86.67

58.72

67.49

78.58

83.78

GPT-5

2026.01

88.3

90.44

89.99

89.61

89.11

86.01

90.41

90.86

86.71

57.47

71.58

79.55

84.17

DeepSeek-R1

2026.01

88.28

89.64

89.63

89.25

87.85

86.69

90.1

89.85

84.13

65.64

69.79

77.36

84.02

Claude-4.0-Sonnet

2026.01

87.78

89.56

89.97

89.32

88.42

87.83

90.32

90.72

80.47

55.56

70.72

76.12

83.07

LLaMA-4-Maverick

2026.01

87.73

89.55

90.02

89.57

87.02

86

89.98

90.05

84.15

54.47

69.27

77.79

82.97

DeepSeek-V3

2026.01

87.64

89.67

88.59

88.74

87.44

84.1

88.48

89.02

77.07

51.05

58.45

64.69

79.58

GPT-4o

2026.01

86.96

87.87

88.95

88.29

87.69

85.07

88.81

89.18

83.97

37.49

61.44

73.68

79.95

gpt-oss-120B

2026.01

86.91

88

88.48

88.13

86.81

85.46

88.13

88.26

77.86

52.62

68.26

76.32

81.27

GPT-4.1-mini

2026.01

86.13

88.59

87.75

87.61

86.53

83.05

88.89

88.41

77.55

27.2

61.77

72.59

78.01

GPT-5-mini

2026.01

85.53

88.36

87.9

87.37

86.53

82.71

88.22

88.34

77.87

33.48

59.4

73.02

78.23

QwQ-32B

2026.01

85.03

86.24

86.57

85.38

82.29

79.6

86.68

86

62.21

23.75

29.49

29.87

68.59

Baichuan-M2-32B

2026.01

84.76

86.1

83.98

82.91

81.52

78.42

83.88

83.72

34.33

25.98

33.35

29.88

65.74

HuatuoGPT-o1-72B

2026.01

84.34

85.54

86.2

84.46

83.03

78.55

86.84

86.85

62.18

44.44

44.91

29.51

71.41

LLaMA-4-Scout

2026.01

84.12

87.1

86.31

86.83

84.23

82.29

86.43

87

77.98

50.25

50.64

69.34

77.71

Qwen3-14B-thinking

2026.01

83.59

86.03

84.71

84.81

83.42

80.83

84.66

85.02

64.36

17.63

24.89

27.53

67.29

Qwen2.5-72B

2026.01

83.21

85.62

84.48

84.17

81.56

78.96

85.1

84.97

54.45

43.9

41.74

43.98

71.01

gpt-oss-20B

2026.01

82.64

78.47

84.91

84.63

82.95

81.27

85.03

84.74

68.96

33.72

61.98

69.13

74.87

MedGemma-27B

2026.01

82.2

86.11

84.85

84.64

81.91

80.72

85.48

85.96

77.75

18.04

50.94

65.88

73.71

Qwen3-8B-thinking

2026.01

81.15

83.88

82.17

82.52

79.96

78.42

81.9

82.18

36.87

14.73

17.74

12.06

61.13

Qwen3-14B

2026.01

80.88

83.93

80.96

80.53

76.98

73.88

80.59

80.92

46.86

15.01

24.17

27.17

62.66

Phi-4-Reasoning

2026.01

79.89

85.5

84.47

84.67

83.64

79.93

78.34

85.09

71.75

17.47

48.91

37.77

69.79

DeepSeek-R1-Qwen3-8B

2026.01

79.34

82.46

77.59

77.49

75.66

73.91

78.48

78.63

36.65

11.08

10.83

13.17

57.94

LLaMA-3.1-70B

2026.01

78.36

85.67

84.26

83.43

77.57

67.09

83.76

84.28

70.4

41.28

45.09

49.39

70.88

Qwen2.5-14B

2026.01

77.17

80.3

76.86

75.79

72.86

68.43

75.99

75.73

37.04

26.76

35.73

35.38

61.5

Claude-3.5-Haiku

2026.01

76.96

83.14

82.79

81.82

79.36

74.99

83.34

83.04

59.26

19.45

42.53

42.95

67.47

Qwen3-8B

2026.01

76.95

80.75

75.54

76.37

72.29

67.65

75.36

75.22

31.28

10.53

22.73

11.42

56.34

GPT-4o-mini

2026.01

76.93

83.54

82.68

81.97

78.1

72.34

82.37

82.18

69.38

35.69

48.67

58.1

71

Gemma-3-27B

2026.01

76.73

82.14

80.95

80.88

77.86

76.83

80.71

81.34

71.33

23.93

52.41

62.27

70.62

Qwen3-4B-thinking

2026.01

76.64

80.4

78.85

78.31

75.53

72.08

77.34

78.18

29.54

17.14

13.21

8.99

57.18

GPT-4.1-nano

2026.01

76.59

84.59

81.86

80.96

74.56

72.63

82.2

81.2

59.13

33.51

44.72

53.54

68.79

Mistral-Small-3.1-24B

2026.01

76.22

83.46

80.23

80.87

74.67

68.63

81.02

81.13

38.85

11.82

13.73

20.57

59.27

HuatuoGPT-o1-7B

2026.01

73.8

69.96

72.11

71.8

65.12

62.83

72.32

70.88

7.32

7.04

7.19

6.26

48.89

Qwen3-4B

2026.01

72.91

76.47

70.4

70.43

66.57

61.1

70

69.67

19.33

14.16

16.25

9.66

51.41

HuatuoGPT-o1-70B

2026.01

72.87

85.19

84.22

84.27

72.69

77.43

84.77

84.83

74.9

43.44

50.15

56.66

72.62

Qwen2.5-7B

2026.01

71.29

74.45

68.09

65.88

63.1

57.78

66.76

66.51

32.25

9.13

24.63

18.55

51.54

Gemma-3-12B

2026.01

70.8

78.31

75.85

75.65

71.93

69.44

75.22

75.47

62.87

11.15

43.19

54.35

63.69

Phi-4

2026.01

70.17

84.6

81.31

81.12

72.99

66.18

80.93

80.43

52.61

23.43

38.22

34.67

63.89

GPT-5-nano

2026.01

67.63

80.66

78.25

77.47

74.14

66.74

75.31

77.05

57.85

17.02

41.53

50.74

63.7

HuatuoGPT-o1-8B

2026.01

64.48

73.33

70.71

70.5

62.24

54.89

69.3

69.15

50.34

9.62

9.04

8.46

51.01

LLaMA-3.3-70B

2026.01

61.85

86.28

85.89

84.91

65.94

76.32

85.67

85.93

72.72

41.81

47.93

51.22

70.54

Qwen2.5-3B

2026.01

61.05

66.95

58.48

55.81

50.25

44.89

56.09

55.37

16.09

9.17

13.37

11.75

41.61

Qwen3-1.7B

2026.01

57.75

65.09

55.49

52.98

45.46

42.18

52.99

52.82

23.75

22.52

26.3

23.17

43.38

LLaMA-3.1-8B

2026.01

56.79

74.27

63.01

62.34

50.87

42.49

62.7

62.16

37.13

23.72

19.75

14.36

47.47

MedGemma-4B

2026.01

55

69.39

63.51

61.79

55.02

50.91

62.94

62.57

45.48

20.66

21.98

30.95

50.02

Gemma-3-4B

2026.01

52.83

64.58

59.58

58.23

53.69

48.51

59.74

59.42

42.89

12.9

25.55

20.01

46.49

Bio-Medical-LLaMA-3-8B

2026.01

50.07

69.06

59.37

58.85

49.4

44.72

57.13

54.77

38.93

35.79

34.01

33.82

48.83

LLaMA-3.2-3B

2026.01

49.2

64.07

53.22

49.62

41.37

31.63

50.68

43.97

33.28

14.19

18.89

17.28

38.95

MedReason-8B

2026.01

47.48

63.47

18.4

14.44

48.26

41.14

26.6

34.88

12.92

8.11

13.19

11.66

28.38

Phi-4-mini

2026.01

42.01

71.17

52.71

55.27

39.4

31.97

50.59

50.39

30.29

14.39

19.51

17.02

39.56

MediPhi

2026.01

38.13

68.04

62.01

61.21

31.25

32.76

57.91

53.8

16.53

14.77

15.83

17.52

39.15

Phi-4-mini-Reasoning

2026.01

36.42

74.05

52.82

60.22

26.83

14.56

49.83

44.06

28.71

20.84

18.5

19.8

37.22

OpenBioLLM-70B

2026.01

31.62

73.19

74.86

52.14

32.53

31.09

58.81

70.78

31.04

10.13

11.79

23.73

41.81

Mistral-7B-v0.3

2026.01

28.39

36.61

21.04

40.89

24.09

27.67

30.07

37.35

18.13

14.69

10.4

9.78

24.93

OpenBioLLM-8B

2026.01

22.73

35.28

24.55

32.01

8.99

12.06

34.79

35.48

11.49

6.95

10.1

11.1

20.46

Multilingual Multiple-Choice Question Answering on HeadQA 1.0 (test)

Evaluation Results