OpenEQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Embodied Question Answering | OpenEQA v1.0 (test) | LLM-Match67.7 | 32 | |
| Episodic-memory Question Answering | OpenEQA v1 (ScanNet) | LLM-Match87.7 | 29 | |
| Embodied Question Answering | OpenEQA EM-EQA | Accuracy86.8 | 19 | |
| Active Question Answering | OpenEQA v1 (HM3D) | LLM-Match85.1 | 12 | |
| Episodic-memory Question Answering | OpenEQA v1 (HM3D) | LLM Match Score85.1 | 12 | |
| Zero-shot understanding of embodied scenes | OpenEQA | Score51.2 | 10 | |
| Embodied Question Answering | OpenEQA (test) | GPT-4 Score86.8 | 10 | |
| Embodied Question Answering | OpenEQA | Score59 | 8 | |
| Embodied Question Answering | OpenEQA EM-EQA | LLM-Match86.8 | 8 | |
| Embodied Question Answering | OpenEQA v1 (test) | Score85.1 | 5 |
Showing 10 of 10 rows