ALFRED
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Instruction Following | ALFRED | Accuracy19.84 | 36 | |
| Embodied AI Task Planning | EB-ALFRED | Average Score70.8 | 28 | |
| Continual Instruction Following | ALFRED | Success Rate (SR)69.9 | 28 | |
| Instruction following | ALFRED (test-unseen) | GC61.6 | 23 | |
| Embodied Instruction Following | ALFRED seen 1.0 (test) | GC54.81 | 20 | |
| Mobile Manipulation | ALFRED (test unseen) | Success Rate (SR)60.79 | 18 | |
| Mobile Manipulation | ALFRED seen (test) | Success Rate (SR)65.09 | 18 | |
| Task Planning | EB-ALFRED (Long) | Success Rate (SR)70 | 17 | |
| Task Progress Estimation | Alfred | pmae2.19 | 15 | |
| Embodied Task Completion | ALFRED unseen (test) | Success Rate4,572 | 14 | |
| Embodied Task Completion | ALFRED seen (test) | Success Rate (SR)53.23 | 14 | |
| Embodied Planning | ALFRED | Success Rate (SR)45.81 | 11 | |
| Embodied AI Task Execution | EB-ALFRED online unsupervised setting | Success Rate (Avg)61 | 10 | |
| Embodied Task Completion | ALFRED EB | Avg Score32.2 | 8 | |
| 3D Instruction Following | ALFRED | Accuracy62 | 8 | |
| Interactive Planning | ALFRED unseen (val) | Success Rate (SR)67.8 | 8 | |
| Interactive Planning | ALFRED (val seen) | SR46.59 | 6 | |
| Delivery (Pick-Place) | ALFRED | TSR71.2 | 4 | |
| Inspection | ALFRED | TSR (%)78.5 | 4 | |
| Navigation | ALFRED | Task Success Rate (TSR)84 | 4 | |
| Action Sequence Generation | ALFRED (val unseen) | Exact Match91 | 4 | |
| High-level Planning | ALFRED (val unseen) | EM64 | 4 | |
| Subtask Completion | ALFRED | Avg Completion Rate0.53 | 4 | |
| Embodied Task Completion | ALFRED Unseen (val) | Task Success Rate (TSR)20 | 3 | |
| Embodied Task Completion | ALFRED seen (val) | Task Success Rate (SR)3.4 | 3 |
Showing 25 of 26 rows