WTQ
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Table Question Answering | WTQ | Accuracy91.25 | 101 | |
| Table Question Answering | WTQ (test) | Denotation Accuracy64.7 | 62 | |
| Question Answering | WTQ (held-in) | Accuracy80.34 | 21 | |
| Question Answering | WTQ | Accuracy60.9 | 21 | |
| Question Answering | WTQ (test) | Accuracy57.11 | 11 | |
| Table Question Answering | ROBUT-WTQ Sentence-Level Paraphrase (test) | Accuracy (Pre-perturbation)81.6 | 10 | |
| Table Question Answering | ROBUT-WTQ Word-Level Paraphrase (test) | Accuracy (Pre-Perturbation)85.8 | 10 | |
| Table Question Answering | ROBUT-WTQ Column Adding (test) | Accuracy (Pre-Perturbation)81.4 | 10 | |
| Table Question Answering | ROBUT-WTQ Column Extension (test) | Accuracy (Pre-perturbation)92 | 10 | |
| Table Question Answering | ROBUT-WTQ Column Order Shuffling (test) | Accuracy (Pre-perturbation)89 | 10 | |
| Table Question Answering | ROBUT-WTQ Abbreviation Replacement (test) | Accuracy (Pre-perturbation)82.9 | 10 | |
| Table Question Answering | ROBUT-WTQ Synonym Replacement (test) | Accuracy (Pre-perturbation)83.6 | 10 | |
| Table Question Answering | WTQ (dev) | Accuracy77.5 | 9 | |
| Text-to-SQL | WTQ (ADVETA-ADD) | Exact Match (EM)44.6 | 8 | |
| Text-to-SQL | WTQ (ADVETA-RPL) | Exact Match (EM)41.8 | 8 | |
| Text-to-SQL | WTQ original (dev) | Exact Match (EM)44.1 | 8 | |
| Text-oriented Visual Question Answering | WTQ | Accuracy31.2 | 6 | |
| Visual Question Answering | WTQ (test) | Accuracy65.4 | 6 | |
| Data-to-text generation | WTQ (test) | ROUGE-162.25 | 5 | |
| Table Question Answering | WTQ Mix ROBUT (test) | Accuracy (Pre-perturbation)64.5 | 5 | |
| Table Question Answering | WTQ Column Masking ROBUT (test) | Accuracy (Pre-Perturbation)60.4 | 5 | |
| Table Question Answering | ROBUT-WTQ (dev) | Accuracy61 | 5 | |
| Table structure recognition | WTQ I (held-in) | TEDS Score73 | 3 | |
| Data-to-text generation | WTQ | FE14.71 | 3 |
Showing 24 of 24 rows