Share your thoughts, 1 month free Claude Pro on usSee more

Spatial Reasoning on SURDS (test)

0Yaw

SpatialBot

Updated 1mo ago

Evaluation Results

Method	Links
SpatialBot 2025.12		0	0	12	0	0	0	2
SpatialRGPT 2025.12		1.3	0.55	10.59	1.95	0.86	7.35	3.77
LLaVA-OV-Qwen2-72B-si 2025.12		1.95	3.03	23.57	3.78	9.73	8.65	8.45
GPT-4o-mini 2025.12		3.24	0.28	0.22	4.22	21.51	2.05	5.25
Random 2025.12		5.73	1.12	34.27	8.76	11.57	11.89	12.22
Qwen2.5-VL-3B 2025.12		6.27	3.81	27.68	17.84	14.81	10.49	13.48
Qwen2.5-VL-3B-Instruct 2026.03		6.27	3.81	27.68	17.84	14.81	10.49	13.48
Qwen2.5VL-7B 2025.12		7.57	3.46	25.95	11.46	17.95	9.3	12.61
Qwen2.5-VL-7B-Instruct 2026.03		7.57	3.46	25.95	11.46	17.95	9.3	12.61
Gemini-2.0-flash 2025.12		9.3	5.41	32.97	69.3	77.3	20	35.71
OmniDrive-R1 2025.12		9.35	39.46	36.72	46.25	46.51	13.42	31.95
Qwen2.5VL-72B 2025.12		11.57	6.13	44	58.05	66.16	14.92	33.47
Qwen2.5-VL-72B-Instruct 2026.03		11.57	6.13	44	58.05	66.16	14.92	33.47
GPT-4o 2025.12		13.08	1.62	2.49	11.57	47.89	3.14	13.3
Gemini-1.5-pro 2025.12		19.14	4.41	22.7	61.95	66.38	22.05	32.77
SURDS-3B* 2026.03		20.97	44.81	69.84	49.3	51.35	8.54	40.8
InternVL3-8B† 2026.03		45.19	61.83	100	82.05	86.16	84.43	76.61
LaST-VLA-8B† 2026.03		70.16	71.28	100	86.05	90.27	88	84.29