ALFRED
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Embodied AI Task Planning | EB-ALFRED | Average Score82 | 87 | |
| Embodied AI Instruction Following | ALFRED (test) | Success Rate (SR)70 | 74 | |
| Instruction Following | ALFRED | Accuracy19.84 | 57 | |
| Embodied Task Completion | ALFRED EB | Avg Score92 | 36 | |
| Instruction following | ALFRED (test-unseen) | GC94.5 | 31 | |
| Continual Instruction Following | ALFRED | Success Rate (SR)69.9 | 28 | |
| Embodied Task Completion | ALFRED unseen (test) | Success Rate4,572 | 26 | |
| Embodied Task Completion | ALFRED seen (test) | Success Rate (SR)53.23 | 26 | |
| Task Planning | EB-ALFRED (Long) | Success Rate (SR)74 | 23 | |
| Embodied Instruction Following | ALFRED seen 1.0 (test) | GC54.81 | 20 | |
| Mobile Manipulation | ALFRED (test unseen) | Success Rate (SR)60.79 | 18 | |
| Mobile Manipulation | ALFRED seen (test) | Success Rate (SR)65.09 | 18 | |
| Task Progress Estimation | Alfred | pmae2.19 | 15 | |
| Skill Evaluation | ALFRED | Object Perception (Grounding) Accuracy78.01 | 12 | |
| Embodied Planning | ALFRED | Success Rate (SR)45.81 | 11 | |
| Embodied AI Task Execution | EB-ALFRED online unsupervised setting | Success Rate (Avg)61 | 10 | |
| Planning | ALFRED Open-ended | Success Rate (SR)80.8 | 8 | |
| Planning | ALFRED (Dynamic) | Success Rate (SR)83.1 | 8 | |
| Planning | ALFRED (Static) | SR88.4 | 8 | |
| 3D Instruction Following | ALFRED | Accuracy62 | 8 | |
| Interactive Planning | ALFRED unseen (val) | Success Rate (SR)67.8 | 8 | |
| Instruction Following | ALFRED seen (test) | Task Success Rate29.16 | 7 | |
| Language-driven scene representation | ALFRED Object Shift [OS] | F1 Score83.92 | 7 | |
| Language-driven scene representation | ALFRED Template Shift [TS] | F1 Score84.9 | 7 | |
| Language-driven scene representation | ALFRED In-Distribution [ID] | F1 Score84.28 | 7 |
Showing 25 of 42 rows