VirtualHome
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Interactive decision making | Virtualhome | Success Rate63.3 | 31 | |
| Embodied Task Planning | VirtualHome | Overall Performance73.5 | 24 | |
| Human-robot teaming for household object rearrangement | VirtualHome 2.0 (test) | Success Rate80 | 19 | |
| Proactive Assistance | VirtualHome v2.2.4 (Apartments #3 and #6) | Speedup (%)38.41 | 18 | |
| Embodied Agent Task | VirtualHome Unseen | Success Rate36.9 | 18 | |
| Embodied Task Planning | VirtualHome (Seen) | Success Rate31.2 | 18 | |
| Robotic Task Planning in Dynamic Environments | VirtualHome | Success Rate92 | 16 | |
| Instruction Execution | VirtualHome Unseen domains | Success Rate83.61 | 15 | |
| Continual Instruction Following | VirtualHome | SR61.12 | 15 | |
| Robotic Task Planning | VirtualHome v2.3.0 | GCR41.9 | 8 | |
| Planning | VirtualHome Open-ended | Success Rate (SR)87.6 | 8 | |
| Planning | VirtualHome Dynamic | Success Rate85.4 | 8 | |
| Planning | VirtualHome Static | Success Rate (SR)90.1 | 8 | |
| Multi-agent coordination | VirtualHome-Social Addition 4→5 | Average Steps48.213 | 6 | |
| Embodied Task Planning | VirtualHome (unseen domains) | Success Rate80.16 | 6 | |
| Few-shot task expansion | VirtualHome average performance (unseen domains) | SR82.59 | 5 | |
| Few-shot task expansion | VirtualHome unseen domains 5-shot | Success Rate83.61 | 5 | |
| Few-shot task expansion | VirtualHome unseen domains 1-shot | SR81.56 | 5 | |
| Embodied Task Planning | VirtualHome Novel Apartment (Unseen) | Simple Success Rate82.9 | 4 | |
| Setuptable | VirtualHome livingroom_and_bedroom | Task Success Rate (TSR)82.7 | 3 | |
| Setuptable | VirtualHome kitchen_and_bedroom | TSR81.9 | 3 | |
| Putfridge | VirtualHome kitchen and bathroom | TSR80.6 | 3 | |
| Putfridge | VirtualHome bathroom_and_livingroom | TSR81.9 | 3 | |
| Preparefood | VirtualHome bedroom_and_bathroom | TSR80.8 | 3 | |
| Preparefood | VirtualHome kitchen and livingroom | TSR80.8 | 3 |
Showing 25 of 28 rows