Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

AI2-THOR

Benchmarks

Task NameDataset NameSOTA ResultTrend
Smart HelpAI2-THOR (Evaluation Environments)
SR59.3
13
Object NavigationAI2-THOR (Simulations)
Success Rate (SR)59.17
12
Embodied Task CompletionAI2-THOR (test)
SR75
11
Visual NavigationAI2-THOR Unseen Scenes (L >= 5) (test)
SPL13.91
11
Object-goal navigationAI2-THOR Offline Data 1 (test)
CF-SR74.4
8
Embodied PlanningAI2-THOR
SR78.35
8
Visual NavigationAI2-THOR L ≥ 5 (val)
SPL14.86
8
Visual NavigationAI2-THOR All (val)
SPL0.1727
8
Decision InferenceAI2-THOR
Success Rate70.2
7
Visual NavigationAI2-THOR Unseen Scenes (All) (test)
SPL16.15
7
Object-goal navigationAI2-THOR Offline Data 2 (test)
CF-SR60.5
6
Object Goal NavigationAI2-THOR Unseen Target Domains v1 (test)
Success Rate (SR)86.4
6
Object Goal NavigationAI2-THOR Seen Target Domains v1 (test)
Success Rate (SR)90.9
6
Object Goal NavigationAI2-THOR Source Domains v1 (train test)
Success Rate (SR)97.9
6
Robotic object searchAI2-THOR Unseen Env., Unseen Goals 1.0
Success Rate (SR)75
5
Robotic object searchAI2-THOR Unseen Env., Seen Goals 1.0
Success Rate (SR)71
5
Robotic object searchAI2-THOR 1.0 (Seen Env., Unseen Goals)
Success Rate (SR)73
5
Robotic object searchAI2-THOR Seen Env., Seen Goals 1.0
Success Rate74
5
Visual NavigationAI2-THOR (test)
SPL17.88
4
2D Visual Quality of Belief Prediction (Observed Scene)AI2-THOR (test)
LPIPS0.0502
3
2D Visual Quality of Belief PredictionAI2-THOR Imagined Scene (test)
FVD271.8
3
Object NavigationAI2-THOR ProcTHOR Habitat-compatible (unseen synthetic test environments)
Success Rate (SR)58.1
3
Robot Task PlanningAI2-THOR Complex tasks
Success Rate78.1
2
Robot Task PlanningAI2-THOR Compound tasks
Success Rate74.5
2
Robot Task PlanningAI2-THOR Simple tasks
Success Rate87.1
2
Showing 25 of 30 rows