Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

WTQ

Benchmarks

Task NameDataset NameSOTA ResultTrend
Table Question AnsweringWTQ
Accuracy91.25
101
Table Question AnsweringWTQ (test)
Denotation Accuracy64.7
62
Tabular AnalysisWTQ
Accuracy51.9
29
Table Question AnsweringWTQ (Evaluation Set)
Alignment Score47
24
Table Question AnsweringWTQ (train)
Alignment Score44
24
Question AnsweringWTQ (held-in)
Accuracy80.34
21
Question AnsweringWTQ
Accuracy60.9
21
Text-oriented Visual Question AnsweringWTQ
Accuracy71.3
12
Question AnsweringWTQ (test)
Accuracy57.11
11
Table Question AnsweringROBUT-WTQ Sentence-Level Paraphrase (test)
Accuracy (Pre-perturbation)81.6
10
Table Question AnsweringROBUT-WTQ Word-Level Paraphrase (test)
Accuracy (Pre-Perturbation)85.8
10
Table Question AnsweringROBUT-WTQ Column Adding (test)
Accuracy (Pre-Perturbation)81.4
10
Table Question AnsweringROBUT-WTQ Column Extension (test)
Accuracy (Pre-perturbation)92
10
Table Question AnsweringROBUT-WTQ Column Order Shuffling (test)
Accuracy (Pre-perturbation)89
10
Table Question AnsweringROBUT-WTQ Abbreviation Replacement (test)
Accuracy (Pre-perturbation)82.9
10
Table Question AnsweringROBUT-WTQ Synonym Replacement (test)
Accuracy (Pre-perturbation)83.6
10
Table Question AnsweringWTQ (dev)
Accuracy77.5
9
Table RetrievalWTQ
Recall@1 (Base)44
8
Text-to-SQLWTQ (ADVETA-ADD)
Exact Match (EM)44.6
8
Text-to-SQLWTQ (ADVETA-RPL)
Exact Match (EM)41.8
8
Text-to-SQLWTQ original (dev)
Exact Match (EM)44.1
8
Visual Question AnsweringWTQ (test)
Accuracy65.4
6
Data-to-text generationWTQ (test)
ROUGE-162.25
5
Table Question AnsweringWTQ Mix ROBUT (test)
Accuracy (Pre-perturbation)64.5
5
Table Question AnsweringWTQ Column Masking ROBUT (test)
Accuracy (Pre-Perturbation)60.4
5
Showing 25 of 28 rows