Our new X account is live! Follow @wizwand_team for updates

Text-to-SQL on Spider 1.0 (test)

87.2EM Acc (Overall)

CHESS+GPT-4

Updated 4d ago

Evaluation Results

Method
CHESS+GPT-4 2024.10	87.2	-	-	-	-	-
CHESS+GPT-4 2024.10	87.2	-	-	-	-	-
DAIL-SQL+GPT-4 2024.10	86.6	-	-	-	-	-
DAIL-SQL+GPT-4 2024.10	86.6	-	-	-	-	-
DAIL-SQL 2023.12	86.6	-	-	-	-	-
DIN-SQL+GPT-4 2024.10	85.3	-	-	-	-	-
DIN-SQL+GPT-4 2024.10	85.3	-	-	-	-	-
DIN-SQL 2023.12	85.3	-	-	-	-	-
MSc-SQL 2024.10	84.7	-	-	-	-	-
MSc-SQL 2024.10	84.7	-	-	-	-	-
MAC-SQL 2023.12	82.8	-	-	-	-	-
C3+ChatGPT 2024.10	82.3	-	-	-	-	-
C3+ChatGPT 2024.10	82.3	-	-	-	-	-
C3 2023.12	82.3	-	-	-	-	-
RESDSQL 2024.10	79.9	-	-	-	-	-
RESDSQL 2024.10	79.9	-	-	-	-	-
NatSQL+T5 2024.10	78	-	-	-	-	-
NatSQL+T5 2024.10	78	-	-	-	-	-
DTS-SQL+Mistral 2024.10	77	-	-	-	-	-
DTS-SQL+Mistral 2024.10	77	-	-	-	-	-
MAC-SQL 2023.12	75.53	-	-	-	-	-
GRAPHIX + PICARD + T5-3B 2023.01	74	-	-	-	-	-
S2SQL + ELECTRA 2023.01	72.1	-	-	-	-	-
LGESQL + ELECTRA 2023.01	72	-	-	-	-	-
PICARD + T5-3B 2023.01	71.9	-	-	-	-	-
Full Fine-tuning (All) 2025.03	71.8	87.5	73.5	63.8	51.8	-
RASAT + PICARD + T5-3B 2023.01	70.9	-	-	-	-	-
MAC-SQL 2023.12	70.58	-	-	-	-	-
SADGA 2021.11	70.1	-	-	-	-	-
RAT-SQL + GAP MODEL 2020.12	69.7	-	-	-	-	-
RATSQL 2021.11	69.7	-	-	-	-	-
SMBOP + GRAPPA 2023.01	69.7	-	-	-	-	-
RAT-SQL + GraPPa 2020.12	69.6	-	-	-	-	-
RATSQL 2021.11	69.6	-	-	-	-	-
RATSQL + GRAPPA 2023.01	69.6	-	-	-	-	-
NatSQL + GAP 2023.01	68.7	-	-	-	-	-
RATSQL 2021.11	68.4	-	-	-	-	-
SADGA 2021.11	66.7	-	-	-	-	-
ShadowGNN 2021.11	66.1	-	-	-	-	-
Full Fine-tuning (S6) 2025.03	65.7	81.9	68.8	58	41	-
RAT-SQL V3 + BERT 2020.12	65.6	-	-	-	-	-
RATSQL 2021.11	65.6	-	-	-	-	-
RATSQL + BERT 2023.01	65.6	-	-	-	-	-
DuoRAT 2021.11	65.4	-	-	-	-	-
YCSQL + BERT 2020.12	65.3	-	-	-	-	-
YCSQL 2021.11	65.3	-	-	-	-	-
RAT-SQL + BART Encoder 2020.12	65.1	-	-	-	-	-
BRIDGE v2 + BERT 2023.01	65	-	-	-	-	-
State-offset Tuning (h) 2025.03	65	89.1	65.9	51.7	40.4	-
ShadowGNN 2020.12	64.8	-	-	-	-	-
RATSQL-HPFT 2021.11	64.4	-	-	-	-	-
LoRA 2025.03	63.9	86.3	68.2	49.4	34.3	-
State-offset Tuning (y) 2025.03	63.1	85.9	64.1	52.3	37.3	-
RAT-SQL V2 + BERT 2020.12	61.9	-	-	-	-	-
AuxNet + BART 2020.12	61.9	-	-	-	-	-
SQL-Llama 2023.12	61.63	-	-	-	-	-
RyanSQL V2 + BERT 2020.12	60.6	-	-	-	-	-
BitFit 2025.03	59.9	82.3	60.8	52.9	31.3	-
Initial State Tuning 2025.03	59.7	82.3	62.3	43.7	35.5	-
RATSQL 2021.11	57.2	-	-	-	-	-
IRNet v2 2021.11	55	-	-	-	-	-
IRNet 2019.09	54.7	-	-	-	-	-
IRNet + BERT 2020.12	54.7	-	-	-	-	-
GNN 2021.11	54.6	-	-	-	-	-
Editing-Based SQL Query Generation 2019.09	53.4	-	-	-	-	-
EditSQL + BERT 2020.12	53.4	-	-	-	-	-
EditSQL 2021.11	53.4	-	-	-	-	-
GAZP + BERT 2023.01	53.3	-	-	-	-	-
Prompt Tuning 2025.03	50.7	75.4	53.8	37.4	19.3	-
IRNet v2 2021.11	48.5	-	-	-	-	-
Global-GNN 2020.12	47.4	-	-	-	-	-
Global-GNN 2021.11	47.4	-	-	-	-	-
IRNet 2019.09	46.7	-	-	-	-	-
Prefix-Tuning 2025.03	45.1	75	45.1	32.2	13.9	-
GNN 2019.09	39.4	-	-	-	-	-
GNN 2021.11	39.4	-	-	-	-	-
Additional-scan 2025.03	35	62	31.9	27.4	12.1	-
Editing-Based SQL Query Generation 2019.09	32.9	-	-	-	-	-
SyntaxSQLNet 2018.10	27.2	48	27	24.3	4.6	-
SyntaxSQLNet 2019.09	27.2	-	-	-	-	-
Lee 2019.09	24.3	-	-	-	-	-
SyntaxSQLNet 2018.10	19.7	38.6	17.6	16.3	4.9	-
SyntaxSQLNet 2019.09	19.7	-	-	-	-	-
SyntaxSQLNet 2018.10	16.4	37.5	13.5	12.4	1.3	-
SQLNet 2018.10	12.4	26.2	12.6	6.6	1.3	-
SQLNet 2019.09	12.4	-	-	-	-	-
TypeSQL 2018.10	8.2	19.6	7.6	3.8	0.8	-
SyntaxSQLNet 2018.10	6.8	18.1	7	0.2	0	-
Seq2Seq+Copying 2018.10	5.3	15.4	3.4	2	1.1	-
Seq2Seq+Attention 2018.10	4.8	14.9	2.5	2	1.1	-
Seq2Seq 2018.10	3.7	11.9	1.9	1.3	0.5	-
RATSQL + GAP + NatSQL 2023.07	-	-	-	-	-	73.3
T5-3B + PICARD 2023.07	-	-	-	-	-	75.1
Graphix-3B + PICARD 2023.07	-	-	-	-	-	77.6
RESDSQL-3B + NatSQL 2023.07	-	-	-	-	-	79.9
DIN-SQL + GPT-4 2023.07	-	-	-	-	-	85.3
C3 + ChatGPT 2023.07	-	-	-	-	-	82.3