GOAT-Bench
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multi-Modal Lifelong Navigation | GOAT-Bench unseen (val) | SR90.6 | 37 | |
| Meme Abuse Detection | GOAT-Bench 1.0 (test) | Harmfulness Accuracy75.1 | 32 | |
| Embodied Navigation | GOAT-Bench unseen (val) | Success Rate (SR)71.4 | 21 | |
| Goal-conditioned navigation | GOAT-Bench | SR73.7 | 21 | |
| Lifelong Visual Navigation | GOAT-Bench 1/10-scale subset (val-unseen) | Success Rate72.8 | 13 | |
| Multi-modal Lifelong Navigation | GOAT-Bench Seen (val) | SPL49 | 13 | |
| Harmful Meme Detection | GOAT-Bench In-Domain | Racism F188.5 | 11 | |
| Lifelong Multimodal Object Navigation | GOAT-Bench unseen (val) | s-SR0.465 | 10 | |
| Subtask Navigation | GOAT-Bench unseen (val) | SR62.7 | 9 | |
| Embodied Navigation | GOAT-Bench | Success Rate (SR)69.1 | 8 | |
| Harmful Meme Detection | GOAT-Bench (Out-Of-Domain) | Racism F187.1 | 7 | |
| Navigation | GOAT-BENCH Strict Physical Regime full 2,669-subtask (val_unseen) | Subtask SR55.6 | 6 | |
| Visual Navigation | Full GOAT-Bench Unseen (val) | SR54.3 | 6 | |
| Visual Navigation | Full GOAT-Bench Synonyms (val) | Success Rate (SR)58.4 | 6 | |
| Visual Navigation | Full GOAT-Bench Seen (val) | SR56.7 | 6 | |
| Multi-modal Lifelong Navigation | GOAT-Bench Seen-Synonyms (val) | SR66.8 | 6 | |
| ObjectNav | GOAT-Bench (val-seen) | SR34.5 | 4 | |
| ImageNav | GOAT-Bench seen (val) | Success Rate (SR)42.2 | 4 | |
| Lifelong Navigation | GOAT-Bench evaluation 278 subtasks | SR72.41 | 4 | |
| Lifelong Navigation | GOAT-Bench full 2,780 subtasks (val) | Success Rate (SR)64.14 | 4 | |
| Object Navigation | GOAT-Bench unseen (val) | SR (%)45 | 4 | |
| Navigation | GOAT-BENCH Relaxed Continuous Regime full 2,669-subtask (val_unseen) | Subtask SR62.9 | 1 |
Showing 22 of 22 rows