Share your thoughts, 1 month free Claude Pro on usSee more

Embodied AI Task Planning on EB-ALFRED

82Average Score

A-Mem

Updated 1mo ago

Evaluation Results

Method
A-Mem 2026.05	82	90	84	86	78	82	72
G-Mem 2026.05	79.27	82	80	78	82	76	78
LangMem 2026.05	78	82	78	84	76	76	72
Mem0 2026.05	77.67	82	82	80	80	74	68
Gemini-1.5-pro 2026.03	75.3	88	80	68	80	72	64
GPT-4o 2026.03	75	82	82	64	78	72	72
Claude-3.5-Sonnet 2026.03	74.7	78	84	66	72	64	84
ELITE 2026.03	70.8	78	78	68	68	62	-
Claude-3.7-Sonnet 2026.04	67.7	68	70	68	68	62	70
Claude-3.7-Sonnet 2026.03	67.7	68	70	68	68	62	70
RoboAgent 2026.04	67	72	64	78	48	60	80
Claude-3.5-Sonnet 2026.03	66.4	72	76	60	66	58	-
ERA (3B) 2026.03	65.2	72	72	62	54	66	-
GPT-4.1 2025.10	64.67	70	70	62	64	62	60
Claude-3.5-Sonnet 2026.03	64	72	76	60	66	58	52
Gemini-1.5-Pro 2026.03	63.2	70	72	58	64	52	-
WAP 2026.04	62.7	66	70	56	62	52	70
Gemini-1.5-Pro 2026.04	62.3	70	72	58	64	52	58
Gemini-1.5-Pro 2026.03	62.3	70	72	58	64	52	58
No Mem 2026.05	59	70	60	68	56	56	44
GPT-4o 2026.03	56.8	64	68	46	54	52	-
GPT-4o 2026.04	56.3	64	68	46	54	52	54
GPT-4o 2026.03	56.3	64	68	46	54	52	54
RoboGPT-R1 2025.10	55.33	62	64	50	56	50	50
RoboGPT-R1 2026.04	55.3	62	64	50	56	50	50
VAGEN (3B) 2026.03	52.8	70	70	58	38	28	-
Gemini-2.0-flash 2025.10	52.3	62	54	46	48	46	58
GPT-4o 2025.10	51.67	54	58	52	46	52	48
Gemini-2.0-flash 2026.03	51.2	62	54	46	48	46	-
RL4VLM (3B) 2026.03	51.2	70	70	56	32	28	-
Vlaser 2026.04	50	-	-	-	-	-	-
Qwen2.5-VL-72B-Ins 2026.03	49.8	55	55	50	42	47	-
LangMem 2026.05	46.67	74	62	46	56	38	4
A-Mem 2026.05	46.33	52	48	34	54	42	48
G-Mem 2026.05	45.33	46	60	42	58	42	24
Qwen2.5-VL-72B-Ins 2025.10	43.67	62	48	40	36	44	32
Qwen-VL-Max 2026.03	43.2	44	44	42	48	38	-
Mem0 2026.05	43.15	48	60	47	58	36	10
Reinforced Reasoner (7B) 2026.03	41.6	54	46	28	42	38	-
Qwen-VL-Max 2026.04	41.3	44	44	42	48	38	32
Qwen-VL-Max 2026.03	41.3	44	44	42	48	38	32
Qwen-VL-Max 2025.10	41.3	44	44	42	48	38	32
MemCompiler 2026.05	40	52	48	38	54	38	10
Qwen2.5-VL-72B 2026.04	39.7	50	42	36	42	34	34
Qwen2.5-VL-72B 2026.03	39.7	50	42	36	42	34	34
InternVL3-78B 2026.03	39.6	38	46	42	34	38	-
Gemini-1.5-flash 2026.03	39.3	44	56	42	40	26	28
gemma-3-27b-it 2026.03	39.2	42	48	30	40	36	-
InternVL3-78B 2026.03	39	38	46	42	34	38	36
LoHo-Manip-4B 2026.04	38	48	54	28	36	30	31
InternVL2_5-78B 2026.03	37.7	38	42	34	34	36	42
InternVL2 5-78B 2026.03	36.8	38	42	34	34	36	-
REBP 2026.04	35.6	54	46	28	42	38	6
MemCompiler 2026.05	35.33	42	44	38	50	34	4
Llama-3.2-90B-Vision 2026.03	35.2	38	44	28	34	32	-
REBP 2025.10	35	52	46	28	46	32	6
Qwen2-VL-72B-Ins 2026.03	34.4	40	40	30	30	32	-
GPT-4o-mini 2025.10	34	66	68	14	56	0	0
A-Mem 2026.05	32.33	46	34	30	50	30	4
Llama-3.2-90B-Vision 2026.03	32	38	44	28	34	32	16
No Mem 2026.05	32	60	28	36	40	28	0
Llama-3.2-90B-Vision-Ins 2025.10	32	38	44	28	34	32	16
Langmem 2026.05	31.33	38	44	28	44	26	8
Mem0 2026.05	30.33	40	40	26	42	26	8
Ovis2-34B 2026.03	29.6	34	38	28	30	18	-
Ovis2-34B 2026.03	28.7	34	38	28	30	18	24
Gemma-3-12b-its 2025.10	25.7	32	38	26	26	20	12
G-Mem 2026.05	25.33	34	38	24	36	14	6
GPT-4o-mini 2026.03	24	34	36	24	28	22	0
GPT-4o mini 2026.04	24	34	36	24	28	22	0
Qwen3-VL-4B 2026.04	19	18	26	14	14	14	26
No Mem 2026.05	19	24	28	24	24	14	0
No Mem 2026.05	18	22	24	12	26	22	2
Mem0 2026.05	14.66	20	26	12	18	10	2
Langmem 2026.05	14.33	22	26	6	20	12	0
MemGen 2026.05	14.33	14	16	6	10	8	0
MemGen 2026.05	14.22	22	16	6	18	20	4
RoboBrain2.0 2026.04	14	-	-	-	-	-	-
Llama3.2-11B 2026.04	14	24	16	22	8	6	6
InternVL3-8B 2026.04	10	20	14	12	14	0	2
G-Mem 2026.05	7.67	4	6	16	14	6	0
A-Mem 2026.05	3	2	4	6	4	2	0
Qwen2.5-VL-7B-Ins 2025.10	2.67	6	6	0	2	2	0
InternVL2.5-8B 2025.10	2	4	2	0	6	0	0
Qwen2.5-VL-3B-Ins 2025.10	1.33	2	0	0	2	4	0
RoboBrain 2025.10	0.33	2	0	0	0	0	0
Tapa 2025.10	0	0	0	0	0	0	0