Share your thoughts, 1 month free Claude Pro on usSee more

Multi-hop spatial reasoning on StepGame (test)

100Accuracy (1-hop)

GPT-5.1

Updated 25d ago

Evaluation Results

Method	Links
GPT-5.1 2026.06		100	93	77	72	57	55	61	49	42	39	64.5
GPT-5.1 2026.06		99	92	85	86	79	71	62	61	50	58	74.3
GPT-5.1 2026.06		98	95	92	91	87	89	86	77	75	70	86
Qwen3-32B 2026.06		94.6	83	75.2	66.4	66.2	62.4	62.4	56.6	59.2	54.2	68
Qwen3-32B 2026.06		94.2	78.2	72.4	69.8	65.4	62.4	63	61	54.2	49.8	67
GPT-5.1 2026.06		94	92	86	86	88	89	90	80	87	78	87
GPT-5.1 2026.06		94	92	87	86	88	89	90	80	87	79	87.2
LLaMA3.1-70B 2026.06		93	62	50	45	36	37	32	34	30	27	44.7
LLaMA3.1-70B 2026.06		92	92	88	87	87	83	88	84	83	82	86.7
GPT-5.1 2026.06		92	92	88	85	86	86	92	85	89	83	87.8
GPT-5.1 2026.06		92	92	88	85	86	88	92	85	89	84	88.1
Qwen3-32B 2026.06		91.8	90.2	86.8	83.6	85.6	81.8	83.6	80.4	78.4	78.4	84.1
Qwen3-32B 2026.06		91.4	87.8	87.8	84.6	86	82.4	84	82	80.6	79.8	84.6
Qwen3-32B 2026.06		89.2	77.4	68.6	63.4	58.2	53.6	56.2	51.6	42.8	45.2	60.6
LLaMA3.1-70B 2026.06		88	65	48	39	35	31	31	30	25	26	42.1
LLaMA3.1-70B 2026.06		88	87	80	73	76	70	77	74	73	70	76.7
Qwen3-32B 2026.06		87.6	80.4	77.6	71.8	77.8	72.4	74.4	71	69.6	73	75.6
LLaMA3.1-70B 2026.06		87	85	84	85	86	80	81	80	78	77	82.3
LLaMA3.1-70B 2026.06		87	80	81	83	78	75	79	76	75	72	78.8
Qwen3-32B 2026.06		85.8	83.6	82.8	81.2	84.4	80.4	82.4	80.6	76.8	77.8	81.6
LLaMA3.1-70B 2026.06		85	79	80	76	74	68	71	68	69	67	73.7