HotPot
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Abstention | Hotpot (test) | AUARC60.9 | 25 | |
| Retrieval-Augmented Generation | Hotpot ENKB5 (In-Domain Average) | Language Consistency86.51 | 15 | |
| Question Answering | ADVHOTPOT | Accuracy82.4 | 12 | |
| Selective Question Answering | HOTPOT | Area under Coverage-F192.5 | 12 | |
| Cross-lingual Retrieval-Augmented Generation | Hotpot-ENKB5 (all) | Language Consistency (ALL-AVG)86.54 | 11 | |
| Retrieval-Augmented Generation | Hotpot-ENKB5 v1 (Indonesian) | Language Consistency69.36 | 11 | |
| Question Answering | HotPot D2 (test) | Macro F196.1 | 11 | |
| Retrieval Question Answering | HotPot | MRR47.7 | 6 | |
| Information Retrieval | Hotpot BEIR | nDCG0.687 | 5 | |
| Multi-hop Question Answering | Hotpot Kimi | EM54.07 | 4 | |
| Retrieval Question Answering | HotPot (in-domain) | MRR63.8 | 4 | |
| Retrieval-Augmented Generation | Hotpot-ENKB5 (Korean) | Language Consistency99.73 | 3 | |
| Error Detection and Recovery | Hotpot Robot Data (test) | Recovery Success Ratio5 | 3 | |
| Human-Robot Interaction | Hotpot | Successes10 | 2 |
Showing 14 of 14 rows