Share your thoughts, 1 month free Claude Pro on usSee more

Spatial Reasoning on SPAR-Bench tiny

72.32Medium Difficulty Score

Human Level

Updated 1mo ago

Evaluation Results

Method	Links
Human Level 2025.12		72.32	67.27	55.31	72.75	74.25	28.75	36.25	78.25	52.25	66.5	33.5	92	64	60.97	76.22	80	94	70	92	80	78	82	50	60
Human Level 2026.03		72.32	67.27	55.31	72.75	74.25	28.75	36.25	78.25	52.25	66.5	33.5	92	64	60.97	76.22	80	94	70	92	80	78	82	50	60
Chance Level (Frequency) 2025.12		38.25	32.74	31.19	43.09	43.51	17.38	13.05	41.9	30.99	27.4	32.17	29.01	26.75	59	32.29	52.94	50.6	28.25	26.92	26.59	26.34	26.74	26.49	25.77
GPT-4o 2025.12		24.93	36.39	29.25	53.8	45	15	13.6	37.4	34.4	23.4	24.4	30	16	28.8	45.11	64	64	58	46	46	32	44	30	22
GPT-4o 2026.03		24.93	36.39	29.25	53.8	45	15	13.6	37.4	34.4	23.4	24.4	30	16	28.8	45.11	64	64	58	46	46	32	44	30	22
Qwen2-VL-72B 2025.12		23.39	35.62	35.28	45.4	49.8	13.8	10	54.6	49.4	36.8	22.4	42	18	10.16	40	60	68	50	38	44	18	28	18	36
Qwen2-VL-72B 2026.03		23.39	35.62	35.28	45.4	49.8	13.8	10	54.6	49.4	36.8	22.4	42	18	10.16	40	60	68	50	38	44	18	28	18	36
Qwen2.5-VL-72B 2025.12		23.05	39.4	35.35	53.2	46.8	17.8	29	49.6	57.4	14.4	14.6	40	16	13.16	48.44	74	74	60	56	50	20	34	24	44
Qwen2.5-VL-72B 2026.03		23.05	39.4	35.35	53.2	46.8	17.8	29	49.6	57.4	14.4	14.6	40	16	13.16	48.44	74	74	60	56	50	20	34	24	44
Chance Level (Random) 2025.12		22.65	-	-	-	-	-	-	-	-	-	-	24.5	-	25.09	23.82	22.02	31.25	25.27	22.16	25.81	24.42	24.17	26.89	-
Claude-3.7-Sonnet 2025.12		7.33	21.77	25.43	41	45.4	11.2	12.2	42.6	19.6	26	5.4	16	6	0	23.33	40	48	22	36	14	12	20	6	12
Claude-3.7-Sonnet 2026.03		7.33	21.77	25.43	41	45.4	11.2	12.2	42.6	19.6	26	5.4	16	6	0	23.33	40	48	22	36	14	12	20	6	12