SOTA Natural Language Inference on BioNLI and PapersWithCode

Method
LLaMA-3.2-3B 2026.01	72.46	45.05	36.17	43.47	60.4	53.49	54.48	55.69	31.2	20.09	27.16	20.14	43.32
o4-mini 2026.01	68.97	73.34	67.13	66.9	68.42	67.08	70.24	71.06	61.42	32.3	49.2	55.24	62.61
gpt-oss-20B 2026.01	68.88	62.77	61.55	60.24	64.22	64.6	62.81	63.91	55.16	24.47	46.64	54.23	57.46
Claude-4.0-Sonnet 2026.01	68.31	75.46	63.88	59.72	62.98	58.72	68.48	67.95	54.74	45.47	47.38	53.82	60.58
GPT-5 2026.01	65.7	73.41	57.73	56.7	63	67.47	60.32	62.98	51.64	48.43	45.86	45.43	58.22
MediPhi 2026.01	64.56	72.46	68.53	73.89	60.16	41.85	72.57	74.08	25.73	44.63	21.35	42.7	55.21
gpt-oss-120B 2026.01	64.07	70.93	59.62	60.13	63.76	64.62	63.1	63.29	67.2	41.49	49.74	58.14	60.51
GPT-4o-mini 2026.01	63.52	63.71	49.23	46.17	53.56	57.05	49.98	50.12	42.23	27.65	40.99	31.44	47.97
Phi-4-Reasoning 2026.01	63.14	69.16	59.47	55.84	61.87	59.33	57.68	61.65	57.74	15.25	48.78	37.89	53.98
Baichuan-M2-32B 2026.01	62.71	57.5	50.36	59.86	58.83	59.71	54.7	55.67	36.48	30.79	41.07	43.31	50.92
DeepSeek-R1 2026.01	62.59	74.17	60.19	60.48	56.87	55.42	64.32	62.49	51.09	50.52	54.59	49.14	58.49
Gemini-2.5-Flash 2026.01	62.44	69.06	58.13	59.4	58.74	61.82	61.29	60.66	53.76	55.54	63.93	54.59	59.95
GPT-5-mini 2026.01	60.99	68.93	60.2	58.04	59.24	59.09	63.31	63.86	53.63	26.32	33.52	52.49	54.97
DeepSeek-R1-Qwen3-8B 2026.01	59.35	64.43	51.34	48.34	52.42	48.93	52.63	54.97	44.67	11.54	18.2	13.92	43.39
GPT-4.1-mini 2026.01	59.1	64.68	55.96	52.49	55.87	54.89	58.95	59.87	46.81	21.39	36.37	37.96	50.36
Qwen2.5-3B 2026.01	58.53	63.52	49.35	46.79	58.77	58.81	55.05	54.3	20.77	1.72	12.86	10.49	40.91
Qwen3-14B 2026.01	58.21	68.72	57.62	52.86	58.35	57.16	63.26	61.61	49.15	2.41	32.63	29.04	49.25
Qwen3-14B-thinking 2026.01	57.79	69.96	59.83	56.71	61.92	60.43	65.14	62.68	55.64	2.6	27.19	33.88	51.15
Mistral-Small-3.1-24B 2026.01	53.63	65.01	48.41	43.97	47.28	48.82	43.9	48.11	42.66	24	21.3	32.49	43.3
Phi-4-mini 2026.01	53.49	54.39	51.75	48.41	54.65	43.9	43.6	49.32	38.82	31.5	31.71	41.1	45.22
HuatuoGPT-o1-72B 2026.01	52.81	64.92	59.08	57.54	53.93	55.59	61.61	63.33	44.26	14.88	40.32	25.41	49.47
Phi-4 2026.01	52.26	59.4	52.55	51.69	53.78	52.24	54.52	54.96	44.45	34.99	44.64	35.97	49.29
Qwen2.5-14B 2026.01	52.19	67.4	48.87	45.29	45.68	48.63	51.31	53.24	45.47	59.04	70.53	38.01	52.14
Qwen3-4B 2026.01	52.17	59.04	40.76	38.76	49.69	43.75	50.2	49.51	11.72	3.21	24.3	14.56	36.47
LLaMA-3.1-8B 2026.01	52.1	38.37	36.39	35.42	68.51	45.4	43.84	40.94	39.99	49.94	38.47	12.67	41.84
QwQ-32B 2026.01	51.84	59.03	49.35	47.15	48.46	47.31	50.77	50.86	47.19	39.39	43.35	45.3	48.33
DeepSeek-V3 2026.01	51.37	63.91	49.8	51.9	49.28	48.66	52.21	50.81	43.59	42.33	37.78	32.85	47.87
Qwen3-1.7B 2026.01	51.29	54.77	45.41	55.56	49.77	41.56	45.1	47.07	49.64	37.22	47.39	42.44	47.27
LLaMA-4-Maverick 2026.01	50.31	61.59	46.13	45.74	50.17	53.14	48.98	49.04	39.67	63.19	42.39	37.39	48.98
Qwen3-4B-thinking 2026.01	50.26	61.98	50.45	47.11	51.75	49.65	54.19	54.57	26.59	3.43	27.4	11.66	40.75
Qwen2.5-7B 2026.01	50.22	62.04	53.87	47.88	49.61	44.7	54.62	50.11	45.43	28.77	38.67	29.16	46.26
GPT-4.1 2026.01	49.67	64.53	51.2	49.07	49.68	52.27	51.45	51.39	40.29	45.09	41.09	34.23	48.33
HuatuoGPT-o1-7B 2026.01	49.02	57.66	52.03	51.43	54.46	49.95	52.49	55.11	16.97	2.74	6.52	4.01	37.7
Qwen3-8B-thinking 2026.01	48.72	62.34	52.52	48.36	56.03	50.5	56.11	56.56	29.98	8.27	29.72	23.39	43.54
LLaMA-4-Scout 2026.01	48.14	56.46	42.52	42.47	43.55	45.14	47.65	44.04	40.84	54.19	62.82	44.04	47.65
Qwen2.5-72B 2026.01	47.66	65	55.39	52.62	50.07	53.59	58.92	60.07	38.96	71.4	55.46	29.4	53.21
Qwen3-8B 2026.01	47.02	61.2	41.1	38.88	51.16	45.59	45.29	48.19	35.02	3.79	52.74	17.08	40.59
LLaMA-3.3-70B 2026.01	46.8	58.4	45.35	41.29	42.4	44.2	46.68	46.01	39	69.33	48.76	29.38	46.47
MedGemma-27B 2026.01	46.39	54.81	45.3	51.22	48.42	52.05	51.33	50.76	48.75	24.67	41.94	46.1	46.81
GPT-4.1-nano 2026.01	46.35	55.66	48.35	45.15	53.86	52.72	50.24	49.21	48.22	46.26	43.89	39.74	48.31
GPT-5-nano 2026.01	45.9	41.14	34.17	45.48	51.23	42.11	49.31	51.51	58.19	31.55	58.94	55.01	47.04
HuatuoGPT-o1-70B 2026.01	44.62	58.75	47.65	46.3	43.58	46.88	48.04	49.35	37.81	55.35	39.05	33.87	45.94
Phi-4-mini-Reasoning 2026.01	43.18	59.28	48.11	44.69	31.63	20.37	37.43	37.06	35.98	45.14	35.04	23.85	38.48
GPT-4o 2026.01	38.77	45.35	42.09	40.1	41.2	47.08	42.96	42.76	34.31	15.88	33.78	34.17	38.2
Claude-3.5-Haiku 2026.01	37.75	50.18	39.77	40.47	40.11	44.79	40.49	40.68	37.09	47.12	70.89	25.76	42.93
Gemma-3-27B 2026.01	36.93	49.26	36.72	39.24	39.79	44	39.22	39.61	35.47	12.51	31.7	37.02	36.79
LLaMA-3.1-70B 2026.01	35.17	58.56	35.9	35.95	36.19	40.67	37.7	35.69	34.54	58.88	32.91	26.15	39.03
Gemma-3-4B 2026.01	35.05	39.87	35.29	34.47	36.01	33.38	34.4	33.8	34.21	2.32	26.58	32.98	31.53
Bio-Medical-LLaMA-3-8B 2026.01	35.03	40.84	25.97	31.76	34.83	25.73	32.19	29.83	26.07	36.28	30.89	35.01	32.04
Mistral-7B-v0.3 2026.01	33.77	2.96	15.37	32.37	26.52	27.15	22.79	19.82	14.17	9.84	10.87	10.78	18.87
HuatuoGPT-o1-8B 2026.01	33.68	34.41	36.26	31.37	44.02	35.35	37.86	37.12	27.49	1.78	21.64	7.56	29.04
Gemma-3-12B 2026.01	32.39	38.32	34.52	32.29	35.82	37.37	36	35.28	39.35	4.83	55.15	35.69	34.75
MedGemma-4B 2026.01	30.72	36.53	39.85	35.28	32.59	36.4	35.67	36.33	32.33	16.32	28.52	28.07	32.38
MedReason-8B 2026.01	27.6	6.41	4.32	4.15	25.53	30.91	24.1	1.69	8.43	0.49	37.09	23.86	16.22
OpenBioLLM-70B 2026.01	12.77	32.56	26.86	15.55	10.26	10.92	22.36	27.53	11.47	11.47	16.16	15.93	17.82
OpenBioLLM-8B 2026.01	9.19	11.68	13.83	17.5	8.92	8.73	17.15	19.13	10.21	14.51	11.25	12.28	12.86

LLaMA-3.2-3B

2026.01

72.46

45.05

36.17

43.47

60.4

53.49

54.48

55.69

31.2

20.09

27.16

20.14

43.32

o4-mini

2026.01

68.97

73.34

67.13

66.9

68.42

67.08

70.24

71.06

61.42

32.3

49.2

55.24

62.61

gpt-oss-20B

2026.01

68.88

62.77

61.55

60.24

64.22

64.6

62.81

63.91

55.16

24.47

46.64

54.23

57.46

Claude-4.0-Sonnet

2026.01

68.31

75.46

63.88

59.72

62.98

58.72

68.48

67.95

54.74

45.47

47.38

53.82

60.58

GPT-5

2026.01

65.7

73.41

57.73

56.7

63

67.47

60.32

62.98

51.64

48.43

45.86

45.43

58.22

MediPhi

2026.01

64.56

72.46

68.53

73.89

60.16

41.85

72.57

74.08

25.73

44.63

21.35

42.7

55.21

gpt-oss-120B

2026.01

64.07

70.93

59.62

60.13

63.76

64.62

63.1

63.29

67.2

41.49

49.74

58.14

60.51

GPT-4o-mini

2026.01

63.52

63.71

49.23

46.17

53.56

57.05

49.98

50.12

42.23

27.65

40.99

31.44

47.97

Phi-4-Reasoning

2026.01

63.14

69.16

59.47

55.84

61.87

59.33

57.68

61.65

57.74

15.25

48.78

37.89

53.98

Baichuan-M2-32B

2026.01

62.71

57.5

50.36

59.86

58.83

59.71

54.7

55.67

36.48

30.79

41.07

43.31

50.92

DeepSeek-R1

2026.01

62.59

74.17

60.19

60.48

56.87

55.42

64.32

62.49

51.09

50.52

54.59

49.14

58.49

Gemini-2.5-Flash

2026.01

62.44

69.06

58.13

59.4

58.74

61.82

61.29

60.66

53.76

55.54

63.93

54.59

59.95

GPT-5-mini

2026.01

60.99

68.93

60.2

58.04

59.24

59.09

63.31

63.86

53.63

26.32

33.52

52.49

54.97

DeepSeek-R1-Qwen3-8B

2026.01

59.35

64.43

51.34

48.34

52.42

48.93

52.63

54.97

44.67

11.54

18.2

13.92

43.39

GPT-4.1-mini

2026.01

59.1

64.68

55.96

52.49

55.87

54.89

58.95

59.87

46.81

21.39

36.37

37.96

50.36

Qwen2.5-3B

2026.01

58.53

63.52

49.35

46.79

58.77

58.81

55.05

54.3

20.77

1.72

12.86

10.49

40.91

Qwen3-14B

2026.01

58.21

68.72

57.62

52.86

58.35

57.16

63.26

61.61

49.15

2.41

32.63

29.04

49.25

Qwen3-14B-thinking

2026.01

57.79

69.96

59.83

56.71

61.92

60.43

65.14

62.68

55.64

2.6

27.19

33.88

51.15

Mistral-Small-3.1-24B

2026.01

53.63

65.01

48.41

43.97

47.28

48.82

43.9

48.11

42.66

24

21.3

32.49

43.3

Phi-4-mini

2026.01

53.49

54.39

51.75

48.41

54.65

43.9

43.6

49.32

38.82

31.5

31.71

41.1

45.22

HuatuoGPT-o1-72B

2026.01

52.81

64.92

59.08

57.54

53.93

55.59

61.61

63.33

44.26

14.88

40.32

25.41

49.47

Phi-4

2026.01

52.26

59.4

52.55

51.69

53.78

52.24

54.52

54.96

44.45

34.99

44.64

35.97

49.29

Qwen2.5-14B

2026.01

52.19

67.4

48.87

45.29

45.68

48.63

51.31

53.24

45.47

59.04

70.53

38.01

52.14

Qwen3-4B

2026.01

52.17

59.04

40.76

38.76

49.69

43.75

50.2

49.51

11.72

3.21

24.3

14.56

36.47

LLaMA-3.1-8B

2026.01

52.1

38.37

36.39

35.42

68.51

45.4

43.84

40.94

39.99

49.94

38.47

12.67

41.84

QwQ-32B

2026.01

51.84

59.03

49.35

47.15

48.46

47.31

50.77

50.86

47.19

39.39

43.35

45.3

48.33

DeepSeek-V3

2026.01

51.37

63.91

49.8

51.9

49.28

48.66

52.21

50.81

43.59

42.33

37.78

32.85

47.87

Qwen3-1.7B

2026.01

51.29

54.77

45.41

55.56

49.77

41.56

45.1

47.07

49.64

37.22

47.39

42.44

47.27

LLaMA-4-Maverick

2026.01

50.31

61.59

46.13

45.74

50.17

53.14

48.98

49.04

39.67

63.19

42.39

37.39

48.98

Qwen3-4B-thinking

2026.01

50.26

61.98

50.45

47.11

51.75

49.65

54.19

54.57

26.59

3.43

27.4

11.66

40.75

Qwen2.5-7B

2026.01

50.22

62.04

53.87

47.88

49.61

44.7

54.62

50.11

45.43

28.77

38.67

29.16

46.26

GPT-4.1

2026.01

49.67

64.53

51.2

49.07

49.68

52.27

51.45

51.39

40.29

45.09

41.09

34.23

48.33

HuatuoGPT-o1-7B

2026.01

49.02

57.66

52.03

51.43

54.46

49.95

52.49

55.11

16.97

2.74

6.52

4.01

37.7

Qwen3-8B-thinking

2026.01

48.72

62.34

52.52

48.36

56.03

50.5

56.11

56.56

29.98

8.27

29.72

23.39

43.54

LLaMA-4-Scout

2026.01

48.14

56.46

42.52

42.47

43.55

45.14

47.65

44.04

40.84

54.19

62.82

44.04

47.65

Qwen2.5-72B

2026.01

47.66

65

55.39

52.62

50.07

53.59

58.92

60.07

38.96

71.4

55.46

29.4

53.21

Qwen3-8B

2026.01

47.02

61.2

41.1

38.88

51.16

45.59

45.29

48.19

35.02

3.79

52.74

17.08

40.59

LLaMA-3.3-70B

2026.01

46.8

58.4

45.35

41.29

42.4

44.2

46.68

46.01

39

69.33

48.76

29.38

46.47

MedGemma-27B

2026.01

46.39

54.81

45.3

51.22

48.42

52.05

51.33

50.76

48.75

24.67

41.94

46.1

46.81

GPT-4.1-nano

2026.01

46.35

55.66

48.35

45.15

53.86

52.72

50.24

49.21

48.22

46.26

43.89

39.74

48.31

GPT-5-nano

2026.01

45.9

41.14

34.17

45.48

51.23

42.11

49.31

51.51

58.19

31.55

58.94

55.01

47.04

HuatuoGPT-o1-70B

2026.01

44.62

58.75

47.65

46.3

43.58

46.88

48.04

49.35

37.81

55.35

39.05

33.87

45.94

Phi-4-mini-Reasoning

2026.01

43.18

59.28

48.11

44.69

31.63

20.37

37.43

37.06

35.98

45.14

35.04

23.85

38.48

GPT-4o

2026.01

38.77

45.35

42.09

40.1

41.2

47.08

42.96

42.76

34.31

15.88

33.78

34.17

38.2

Claude-3.5-Haiku

2026.01

37.75

50.18

39.77

40.47

40.11

44.79

40.49

40.68

37.09

47.12

70.89

25.76

42.93

Gemma-3-27B

2026.01

36.93

49.26

36.72

39.24

39.79

44

39.22

39.61

35.47

12.51

31.7

37.02

36.79

LLaMA-3.1-70B

2026.01

35.17

58.56

35.9

35.95

36.19

40.67

37.7

35.69

34.54

58.88

32.91

26.15

39.03

Gemma-3-4B

2026.01

35.05

39.87

35.29

34.47

36.01

33.38

34.4

33.8

34.21

2.32

26.58

32.98

31.53

Bio-Medical-LLaMA-3-8B

2026.01

35.03

40.84

25.97

31.76

34.83

25.73

32.19

29.83

26.07

36.28

30.89

35.01

32.04

Mistral-7B-v0.3

2026.01

33.77

2.96

15.37

32.37

26.52

27.15

22.79

19.82

14.17

9.84

10.87

10.78

18.87

HuatuoGPT-o1-8B

2026.01

33.68

34.41

36.26

31.37

44.02

35.35

37.86

37.12

27.49

1.78

21.64

7.56

29.04

Gemma-3-12B

2026.01

32.39

38.32

34.52

32.29

35.82

37.37

36

35.28

39.35

4.83

55.15

35.69

34.75

MedGemma-4B

2026.01

30.72

36.53

39.85

35.28

32.59

36.4

35.67

36.33

32.33

16.32

28.52

28.07

32.38

MedReason-8B

2026.01

27.6

6.41

4.32

4.15

25.53

30.91

24.1

1.69

8.43

0.49

37.09

23.86

16.22

OpenBioLLM-70B

2026.01

12.77

32.56

26.86

15.55

10.26

10.92

22.36

27.53

11.47

16.16

15.93

17.82

OpenBioLLM-8B

2026.01

9.19

11.68

13.83

17.5

8.92

8.73

17.15

19.13

10.21

14.51

11.25

12.28

12.86

Natural Language Inference on BioNLI

Evaluation Results