Share your thoughts, 1 month free Claude Pro on usSee more

Linguistic Minimal Pair Evaluation on BLiMP (test)

100NPI lic. (2)

Sonnet

Updated 1mo ago

Evaluation Results

Method	Links
Sonnet 2025.06		100	96	98	98.4	62	85.6	81.3	100	90.2
Sonnet 2025.06		99.3	88.7	91	93.8	98	97.6	71.6	98	92.2
Sonnet 2025.06		99.3	96.7	99	95.8	98	97.8	74.9	100	95.2
gpt-4o 2025.06		98.7	84.7	97.7	97.3	59.3	84.7	80.5	100	87.9
Llama (GPb+CT) 2025.06		98.3	76.4	90.3	91.3	87.5	97.6	76.5	91.3	88.7
gpt-4o 2025.06		97	88.7	92.3	94.7	92	95.6	81.6	100	92.7
Haiku 2025.06		96.7	86	70.7	90.4	87.3	94.7	74.3	91.7	86.5
Sonnet 2025.06		96.7	96	98	93.8	77	90.4	72.8	97.7	90.3
Haiku 2025.06		96	82.7	74.3	94.2	73.7	73.1	73.3	91.3	82.3
gpt-4o 2025.06		93.7	86.7	77.7	94.9	88.7	94.4	82.1	99.3	89.7
gpt-3.5 2025.06		88.7	74.7	68.7	94.4	72	59.6	74.4	91	77.9
Llama (GPb) 2025.06		87.7	70.7	85.3	87.6	69.3	94.4	71.9	81	81
gpt-4o 2025.06		87.7	90.7	80.3	92.4	69.7	81.3	79.1	100	85.1
gpt-3.5 2025.06		87	66	62	82.9	79.7	86	70.9	92.7	78.4
gpt-4o 2025.06		81.7	88	90	89.8	85.7	80.7	77.1	86.3	84.9
Sonnet 2025.06		79	95.3	91	88.2	88	71.3	65.9	63.7	80.3
Haiku 2025.06		77	86.7	64.7	75.1	59.3	62.4	60.5	79	70.6
Sonnet 2025.06		75.7	95.3	90.7	84.7	81	80.2	68.8	62.3	79.8
Haiku 2025.06		73.3	85.3	63.7	83.1	75	62.9	64.7	68.3	72
gpt-4o 2025.06		73.3	88	72.7	86	70.3	75.3	77.2	81	78
gpt-3.5 2025.06		69.7	79.3	67.7	92.9	72.7	58.9	65.1	83	73.6
gpt-3.5 2025.06		67.3	66	57.3	79.1	69.7	48.4	61.6	52	62.7
gpt-3.5 2025.06		67.3	46.7	57.3	87.8	74	88.9	63.1	95	72.5
Llama (CT) 2025.06		66.7	82.6	81	88	70.3	61.3	70.8	57.1	72.2
Haiku 2025.06		65.3	74	50.7	70.7	50	87.6	59.1	81.3	67.3
Llama (Base) 2025.06		62.3	80.7	87.7	85.3	54	55.7	65.1	60.3	68.9
gpt-3.5 2025.06		60	77.3	65.3	83.1	76	52.7	68.9	59.7	67.9
Haiku 2025.06		57.3	74.7	56.7	69.3	60.3	55.6	60.9	54.7	61.2