Share your thoughts, 1 month free Claude Pro on usSee more

Text-to-SQL on Spider 1.0 (test)

91.2EM Acc (Overall)

SIRIUS-SQL

Updated 1d ago

Evaluation Results

Method
SIRIUS-SQL 2026.05	91.2	-	-	-	-	-
SIRIUS-SQL 2026.05	90.41	-	-	-	-	-
SIRIUS 2026.05	89.99	-	-	-	-	-
SIRIUS-SQL 2026.05	89.85	-	-	-	-	-
OmniSQL 2026.05	89.8	-	-	-	-	-
DeepEye-SQL 2026.05	89.8	-	-	-	-	-
XiYan-SQL 2026.05	89.65	-	-	-	-	-
MCS-SQL 2026.05	89.6	-	-	-	-	-
Arctic-Text2SQL-R1 2026.05	88.7	-	-	-	-	-
Gemini-3.1 Pro 2026.05	88.02	-	-	-	-	-
DeepEye-SQL 2026.05	87.61	-	-	-	-	-
CHASE-SQL 2026.05	87.6	-	-	-	-	-
CHESS+GPT-4 2024.10	87.2	-	-	-	-	-
CHESS+GPT-4 2024.10	87.2	-	-	-	-	-
OpenSearch-SQL 2026.05	87.1	-	-	-	-	-
DAIL-SQL+GPT-4 2024.10	86.6	-	-	-	-	-
DAIL-SQL+GPT-4 2024.10	86.6	-	-	-	-	-
DAIL-SQL 2023.12	86.6	-	-	-	-	-
DAIL-SQL 2026.05	86.6	-	-	-	-	-
Qwen3-235B 2026.05	86.21	-	-	-	-	-
DIN-SQL+GPT-4 2024.10	85.3	-	-	-	-	-
DIN-SQL+GPT-4 2024.10	85.3	-	-	-	-	-
DIN-SQL 2023.12	85.3	-	-	-	-	-
DIN-SQL 2026.05	85.3	-	-	-	-	-
Qwen3-Coder 2026.05	85.1	-	-	-	-	-
MSc-SQL 2024.10	84.7	-	-	-	-	-
MSc-SQL 2024.10	84.7	-	-	-	-	-
GPT-4o 2026.05	83.2	-	-	-	-	-
MAC-SQL 2023.12	82.8	-	-	-	-	-
C3+ChatGPT 2024.10	82.3	-	-	-	-	-
C3+ChatGPT 2024.10	82.3	-	-	-	-	-
C3 2023.12	82.3	-	-	-	-	-
Reasoning-SQL 2026.05	81.43	-	-	-	-	-
RESDSQL 2024.10	79.9	-	-	-	-	-
RESDSQL 2024.10	79.9	-	-	-	-	-
NatSQL+T5 2024.10	78	-	-	-	-	-
NatSQL+T5 2024.10	78	-	-	-	-	-
DTS-SQL+Mistral 2024.10	77	-	-	-	-	-
DTS-SQL+Mistral 2024.10	77	-	-	-	-	-
MAC-SQL 2023.12	75.53	-	-	-	-	-
GRAPHIX + PICARD + T5-3B 2023.01	74	-	-	-	-	-
S2SQL + ELECTRA 2023.01	72.1	-	-	-	-	-
LGESQL + ELECTRA 2023.01	72	-	-	-	-	-
PICARD + T5-3B 2023.01	71.9	-	-	-	-	-
Full Fine-tuning (All) 2025.03	71.8	87.5	73.5	63.8	51.8	-
RASAT + PICARD + T5-3B 2023.01	70.9	-	-	-	-	-
MAC-SQL 2023.12	70.58	-	-	-	-	-
SADGA 2021.11	70.1	-	-	-	-	-
RAT-SQL + GAP MODEL 2020.12	69.7	-	-	-	-	-
RATSQL 2021.11	69.7	-	-	-	-	-
SMBOP + GRAPPA 2023.01	69.7	-	-	-	-	-
RAT-SQL + GraPPa 2020.12	69.6	-	-	-	-	-
RATSQL 2021.11	69.6	-	-	-	-	-
RATSQL + GRAPPA 2023.01	69.6	-	-	-	-	-
NatSQL + GAP 2023.01	68.7	-	-	-	-	-
RATSQL 2021.11	68.4	-	-	-	-	-
SADGA 2021.11	66.7	-	-	-	-	-
ShadowGNN 2021.11	66.1	-	-	-	-	-
Full Fine-tuning (S6) 2025.03	65.7	81.9	68.8	58	41	-
RAT-SQL V3 + BERT 2020.12	65.6	-	-	-	-	-
RATSQL 2021.11	65.6	-	-	-	-	-
RATSQL + BERT 2023.01	65.6	-	-	-	-	-
DuoRAT 2021.11	65.4	-	-	-	-	-
YCSQL + BERT 2020.12	65.3	-	-	-	-	-
YCSQL 2021.11	65.3	-	-	-	-	-
RAT-SQL + BART Encoder 2020.12	65.1	-	-	-	-	-
BRIDGE v2 + BERT 2023.01	65	-	-	-	-	-
State-offset Tuning (h) 2025.03	65	89.1	65.9	51.7	40.4	-
ShadowGNN 2020.12	64.8	-	-	-	-	-
RATSQL-HPFT 2021.11	64.4	-	-	-	-	-
LoRA 2025.03	63.9	86.3	68.2	49.4	34.3	-
State-offset Tuning (y) 2025.03	63.1	85.9	64.1	52.3	37.3	-
RAT-SQL V2 + BERT 2020.12	61.9	-	-	-	-	-
AuxNet + BART 2020.12	61.9	-	-	-	-	-
SQL-Llama 2023.12	61.63	-	-	-	-	-
RyanSQL V2 + BERT 2020.12	60.6	-	-	-	-	-
BitFit 2025.03	59.9	82.3	60.8	52.9	31.3	-
Initial State Tuning 2025.03	59.7	82.3	62.3	43.7	35.5	-
RATSQL 2021.11	57.2	-	-	-	-	-
IRNet v2 2021.11	55	-	-	-	-	-
IRNet 2019.09	54.7	-	-	-	-	-
IRNet + BERT 2020.12	54.7	-	-	-	-	-
GNN 2021.11	54.6	-	-	-	-	-
Editing-Based SQL Query Generation 2019.09	53.4	-	-	-	-	-
EditSQL + BERT 2020.12	53.4	-	-	-	-	-
EditSQL 2021.11	53.4	-	-	-	-	-
GAZP + BERT 2023.01	53.3	-	-	-	-	-
Prompt Tuning 2025.03	50.7	75.4	53.8	37.4	19.3	-
IRNet v2 2021.11	48.5	-	-	-	-	-
Global-GNN 2020.12	47.4	-	-	-	-	-
Global-GNN 2021.11	47.4	-	-	-	-	-
IRNet 2019.09	46.7	-	-	-	-	-
Prefix-Tuning 2025.03	45.1	75	45.1	32.2	13.9	-
GNN 2019.09	39.4	-	-	-	-	-
GNN 2021.11	39.4	-	-	-	-	-
Additional-scan 2025.03	35	62	31.9	27.4	12.1	-
Editing-Based SQL Query Generation 2019.09	32.9	-	-	-	-	-
SyntaxSQLNet 2018.10	27.2	48	27	24.3	4.6	-
SyntaxSQLNet 2019.09	27.2	-	-	-	-	-
Lee 2019.09	24.3	-	-	-	-	-

Showing 100 of 116 rows