Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

WTQ

Benchmarks

Task NameDataset NameSOTA ResultTrend
Table Question AnsweringWTQ
Accuracy91.25
101
Table Question AnsweringWTQ (test)
Denotation Accuracy64.7
45
Question AnsweringWTQ (test)
Accuracy57.11
11
Question AnsweringWTQ
Accuracy31.63
11
Table Question AnsweringROBUT-WTQ Sentence-Level Paraphrase (test)
Accuracy (Pre-perturbation)81.6
10
Table Question AnsweringROBUT-WTQ Word-Level Paraphrase (test)
Accuracy (Pre-Perturbation)85.8
10
Table Question AnsweringROBUT-WTQ Column Adding (test)
Accuracy (Pre-Perturbation)81.4
10
Table Question AnsweringROBUT-WTQ Column Extension (test)
Accuracy (Pre-perturbation)92
10
Table Question AnsweringROBUT-WTQ Column Order Shuffling (test)
Accuracy (Pre-perturbation)89
10
Table Question AnsweringROBUT-WTQ Abbreviation Replacement (test)
Accuracy (Pre-perturbation)82.9
10
Table Question AnsweringROBUT-WTQ Synonym Replacement (test)
Accuracy (Pre-perturbation)83.6
10
Table Question AnsweringWTQ (dev)
Accuracy77.5
9
Text-to-SQLWTQ (ADVETA-ADD)
Exact Match (EM)44.6
8
Text-to-SQLWTQ (ADVETA-RPL)
Exact Match (EM)41.8
8
Text-to-SQLWTQ original (dev)
Exact Match (EM)44.1
8
Text-oriented Visual Question AnsweringWTQ
Accuracy31.2
6
Visual Question AnsweringWTQ (test)
Accuracy65.4
6
Data-to-text generationWTQ (test)
ROUGE-162.25
5
Table Question AnsweringWTQ Mix ROBUT (test)
Accuracy (Pre-perturbation)64.5
5
Table Question AnsweringWTQ Column Masking ROBUT (test)
Accuracy (Pre-Perturbation)60.4
5
Table Question AnsweringROBUT-WTQ (dev)
Accuracy61
5
Data-to-text generationWTQ
FE14.71
3
Showing 22 of 22 rows