Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

AI2-THOR

Benchmarks

Task NameDataset NameSOTA ResultTrend
Smart HelpAI2-THOR (Evaluation Environments)
SR59.3
13
Visual NavigationAI2-THOR Unseen Scenes (L >= 5) (test)
SPL13.91
11
Visual NavigationAI2-THOR L ≥ 5 (val)
SPL14.86
8
Visual NavigationAI2-THOR All (val)
SPL0.1727
8
Decision InferenceAI2-THOR
Success Rate70.2
7
Visual NavigationAI2-THOR Unseen Scenes (All) (test)
SPL16.15
7
Object Goal NavigationAI2-THOR Unseen Target Domains v1 (test)
Success Rate (SR)86.4
6
Object Goal NavigationAI2-THOR Seen Target Domains v1 (test)
Success Rate (SR)90.9
6
Object Goal NavigationAI2-THOR Source Domains v1 (train test)
Success Rate (SR)97.9
6
Robotic object searchAI2-THOR Unseen Env., Unseen Goals 1.0
Success Rate (SR)75
5
Robotic object searchAI2-THOR Unseen Env., Seen Goals 1.0
Success Rate (SR)71
5
Robotic object searchAI2-THOR 1.0 (Seen Env., Unseen Goals)
Success Rate (SR)73
5
Robotic object searchAI2-THOR Seen Env., Seen Goals 1.0
Success Rate74
5
Visual NavigationAI2-THOR (test)
SPL17.88
4
Robot Task PlanningAI2-THOR Complex tasks
Success Rate78.1
2
Robot Task PlanningAI2-THOR Compound tasks
Success Rate74.5
2
Robot Task PlanningAI2-THOR Simple tasks
Success Rate87.1
2
Robot Task PlanningAI2-THOR Elemental tasks
Success Rate100
2
Bring a mug of coffee to the tableAI2-THOR (test)
Metric-
0
Set a place at the dining tableAI2-THOR (test)
Metric-
0
Soak the mugAI2-THOR (test)
Metric-
0
Put the apple in the fridgeAI2-THOR (test)
Metric-
0
Showing 22 of 22 rows