Share your thoughts, 1 month free Claude Pro on usSee more

Embodied Task Completion on EB-Habitat

77Avg Success Rate

Claude-4.6

Updated 7d ago

Evaluation Results

Method
Claude-4.6 2026.05	77	100	96	76	68	46	76	-	-	-	-	-	-	-
Claude-3.7 2026.05	76	100	94	82	66	43	70	-	-	-	-	-	-	-
GPT-5 2026.05	75	98	82	72	76	46	74	-	-	-	-	-	-	-
Claude-3.5-Sonnet 2026.03	73	96	74	84	46	64	74	-	-	-	-	-	-	-
Claude-3.5-Sonnet 2026.03	68	96	68	78	38	58	70	-	-	-	-	-	-	-
Claude-3.7 2026.05	67	98	78	76	40	43	68	-	-	-	-	-	-	-
Claude-4.6 2026.05	67	98	72	70	40	46	74	-	-	-	-	-	-	-
Gemini-1.5-pro 2026.03	64.3	94	60	58	46	60	68	-	-	-	-	-	-	-
GPT-4o 2026.05	64	96	72	60	62	40	52	-	-	-	-	-	-	-
GPT-4o 2026.03	63.3	90	46	60	42	68	74	-	-	-	-	-	-	-
ReAct 2026.01	61.2	90	58	58	38	-	62	70.8	97.5	68.5	79.5	72	43.8	-
GPT-4o 2026.03	59	86	44	56	36	64	68	-	-	-	-	-	-	-
Claude-3.7-Sonnet 2026.03	58.7	90	58	58	38	46	62	-	-	-	-	-	-	-
ReAct 2026.01	58	86	44	56	36	-	68	70.7	90.7	56	68	75.2	62.1	-
ReAct 2026.01	57.2	92	52	48	38	-	56	61	92.5	53.5	49.5	59.4	50	-
GPT-5 2026.05	57	84	42	64	40	46	66	-	-	-	-	-	-	-
Gemini-1.5-Pro 2026.03	56.3	92	52	48	38	52	56	-	-	-	-	-	-	-
InternVL3-78B 2026.03	55	84	58	60	32	40	56	-	-	-	-	-	-	-
WorldMind 2026.01	50.8	86	44	44	34	-	46	57.2	87	48.5	51.5	52.3	46.7	-
InternVL2_5-78B 2026.03	49	80	42	56	30	28	58	-	-	-	-	-	-	-
Qwen2.5-32B 2026.05	49	90	46	58	56	18	28	-	-	-	-	-	-	-
WorldMind 2026.01	48.8	86	42	44	32	-	40	56.7	89	47	51	49.8	46.8	-
SimuRA 2026.01	48.4	84	42	40	32	-	44	55.4	85	45.5	47	48.2	51.5	-
SimuRA 2026.01	48	82	40	36	38	-	44	55.3	83	47	40	48.9	57.8	-
GPT-4o 2026.05	48	92	32	50	30	40	46	-	-	-	-	-	-	-
ReAct 2026.01	47.6	84	30	58	32	-	34	55.2	82	43	59	63.9	45.1	-
Synapse 2026.01	47.2	84	40	38	38	-	42	54.2	85	43	43	46.3	53.5	-
ReasoningBank 2026.01	46.4	76	40	40	34	-	42	53.9	78.5	41.5	44	46.2	59.5	-
AWM 2026.01	46.4	78	42	44	32	-	36	50.9	78	43	48	39.7	45.7	-
ReasoningBank 2026.01	46.4	76	42	30	32	-	52	53.7	77	45.5	34.5	58.4	53.3	-
ReAct 2026.01	45.6	94	24	50	28	-	32	50.6	94.5	32.5	53	39.7	59.6	-
Synapse 2026.01	45.6	78	42	38	36	-	34	52	78.3	46.5	45.5	39.7	50	-
Qwen-VL-Max 2026.03	45.3	74	40	50	30	36	42	-	-	-	-	-	-	-
BoN 2026.01	44.4	74	32	36	34	-	46	50.9	75.8	38	41.5	48.9	50.5	-
Qwen3-32B 2026.05	44	88	46	46	66	6	14	-	-	-	-	-	-	-
ReAct 2026.01	43.6	82	30	34	34	-	38	50.4	84	34	40	43.4	50.4	-
BoN 2026.01	43.6	74	34	36	34	-	40	50.8	75.5	35.5	43	46.9	53.3	-
AWM 2026.01	43.6	74	32	38	30	-	44	50.8	74.8	37	44.5	47.9	49.7	-
ReAct 2026.01	41.6	74	26	32	36	-	40	47.4	76	30	37	43.7	50.3	-
Llama-3.2-90B-Vision 2026.03	40.3	94	24	50	28	14	32	-	-	-	-	-	-	-
Gemini-1.5-flash 2026.03	39.3	76	32	48	32	12	36	-	-	-	-	-	-	-
Qwen3-8B 2026.05	39	74	38	38	46	12	24	-	-	-	-	-	-	-
Qwen2.5-VL-72B 2026.03	37.7	74	28	42	24	18	40	-	-	-	-	-	-	-
Ovis2-34B 2026.03	37	68	34	38	30	14	38	-	-	-	-	-	-	-
ReAct 2026.01	36.4	74	22	32	32	-	22	44	77.5	32.5	42	33.1	57.8	-
Qwen2.5-32B 2026.05	34	68	26	48	20	18	26	-	-	-	-	-	-	-
Gemma-3-12B-IT + µOnline RL 2026.01	33.8	60	13	37	35	24	-	-	-	-	-	-	-	-
GPT-4o-mini 2026.03	32.7	74	22	32	32	14	22	-	-	-	-	-	-	-
Gemma-3-12B-IT + µOffline Sup. 2026.01	31.8	66	14	35	27	17	-	-	-	-	-	-	-	-
Gemma-3-12B-IT + µSimple 2026.01	31.2	62	15	30	31	18	-	-	-	-	-	-	-	-
Qwen3-32B 2026.05	30	78	22	30	36	6	10	-	-	-	-	-	-	-
Qwen3.5-35B 2026.05	26	52	22	26	46	2	10	-	-	-	-	-	-	-
Gemma-3-12B-IT 2026.01	23	58	10	24	24	4	-	-	-	-	-	-	-	-
Qwen3-8B 2026.05	23	44	14	24	18	12	24	-	-	-	-	-	-	-
Qwen2.5-VL-7B-Ins + µOffline Sup. 2026.01	22.8	39	5	33	17	20	-	-	-	-	-	-	-	-
Qwen2.5-VL-7B-Ins + µOnline RL 2026.01	22.2	37	3	37	16	18	-	-	-	-	-	-	-	-
Qwen2.5-VL-7B-Ins + µSimple 2026.01	21.4	39	10	31	14	13	-	-	-	-	-	-	-	-
Argos 2025.12	20.7	45.3	12	24	17.3	9.3	16	-	-	-	-	-	-	-
Video-R1 2025.12	16.2	46.7	6	15.3	16.7	3.3	9.3	-	-	-	-	-	-	-
Qwen2.5-VL-7B-Ins 2026.01	14.3	32	2	26	14	2	-	-	-	-	-	-	-	-
Qwen2.5-VL-7B 2025.12	12.6	30.7	6.7	13.3	12	1.3	11.3	-	-	-	-	-	-	-
Qwen3.5-35B 2026.05	11	30	6	12	10	2	6	-	-	-	-	-	-	-
Qwen2.5-VL-7B 2025.12	9	23.3	5.3	10.7	8	0	6.7	-	-	-	-	-	-	-
GPT-4o 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	59
Qwen-VL-Max 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	45.3
Qwen2.5-VL-7B-Inst. 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	14.3
InternVL3-8B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	24.3
InternVL3.5-8B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	32
Qwen3-VL-8B-Inst. 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	27.7
RoboBrain2.0-7B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	29.3
RoboBrain2.5-8B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	26.3
VeBrain-7B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	15
Pelican-VL-7B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	16.3
MiMo-Embodied-7B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	16.7
Vlaser-8B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	40
ACE-Brain-0-8B 2026.03	-	-	-	-	-	-	-	-	-	-	-	-	-	42.3