GrailQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multi-Hop Knowledge Graph Question Answering | GrailQA | Hits@194.4 | 68 | |
| Knowledge Base Question Answering | GrailQA Freebase (test) | Hits@186.9 | 48 | |
| Knowledge Base Question Answering | GrailQA | Hits@189.3 | 40 | |
| Knowledge Base Question Answering | GrailQA (dev) | Overall F189.5 | 34 | |
| Knowledge Base Question Answering | GrailQA v1.0 (test) | Overall EM77.45 | 33 | |
| Knowledge Base Question Answering | GrailQA (test) | F191.76 | 27 | |
| Knowledge Graph Question Answering | GrailQA (Overall) | Hits@186.4 | 20 | |
| Knowledge Base Question Answering | GrailQA 500-sample (dev) | F1 Score84.7 | 18 | |
| Knowledge Graph Question Answering | GrailQA Zero-shot | Hits@189.1 | 17 | |
| Knowledge Graph Question Answering | GrailQA Compositional | Hits@180 | 17 | |
| Knowledge Graph Question Answering | GrailQA I.I.D. | Hits@192 | 17 | |
| Knowledge Base Question Answering | GrailQA Compositional (dev) | F1 Score87 | 14 | |
| Knowledge Base Question Answering | GrailQA i.i.d. (dev) | F193.8 | 14 | |
| Knowledge Graph Question Answering | GrailQA (test) | Overall Score84.7 | 14 | |
| Knowledge Base Question Answering | GrailQA | Hits@180.4 | 11 | |
| Generalization Knowledge Base Question Answering | GrailQA | Hit@189.3 | 9 | |
| Knowledge Base Question Answering | GrailQA v1.0 (dev) | F183.4 | 9 | |
| Knowledge Base Question Answering | GrailQAbility answerable zero-shot | F1 (L)78.01 | 8 | |
| Knowledge Base Question Answering | GrailQAbility answerable (IID) | F1(L)89 | 8 | |
| Knowledge Graph Question Answering | GrailQA IID | F1 Score92.4 | 6 | |
| Knowledge Base Question Answering | GrailQAbility unanswerable questions Zero-Shot (test) | F1(R)88.31 | 4 | |
| Knowledge Base Question Answering | GrailQAbility unanswerable questions (test IID) | F1(R)97.01 | 4 | |
| Knowledge Base Question Answering | GrailQA (val) | Overall F183.33 | 4 | |
| Knowledge Base Question Answering | GrailQA (hard) | EM51.5 | 3 |
Showing 24 of 24 rows