VLABench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Success Rate Evaluation | VLABench | Average Success Rate46.3 | 19 | |
| Failure Detection | VLABench (Unseen Tasks) | bACC71.3 | 12 | |
| Failure Detection | VLABench (Seen Tasks) | Balanced Accuracy (bACC)85.6 | 12 | |
| Robot Manipulation | VLABench | Toy Success Rate70 | 5 | |
| Language-conditioned visual reasoning | VLABench official (test) | Precision Score (Toy)76 | 4 | |
| Robotic Task Planning | VLABench | Toy Success Rate54 | 4 | |
| Language-conditioned visual reasoning | VLABench | SR (Toy)54 | 4 | |
| Robotic Manipulation Reasoning | VLABench | In Distribution Accuracy54 | 3 | |
| Robotic Manipulation | VLABench 5 public tracks v1.0 | IS (In-dist)79.8 | 3 | |
| Robot Manipulation | VLABench Cross Category | Add Condiment Success Rate14 | 2 | |
| Robot Manipulation | VLABench In Distribution | Add Condiment Success63 | 2 |
Showing 11 of 11 rows