Multi-turn Text-to-SQL on SparC (dev)

68.5QM EM

gpt-oss-20b + Rose-SQL

Updated 28d ago

Evaluation Results

Method
gpt-oss-20b + Rose-SQL 2026.05	68.5	80.25	75.3	52.4	65.4	59.5
RASAT + PICARD 2026.03	67.7	73.3	-	49.1	54	-
RASAT + PICARD 2026.05	67.7	73.3	-	49.1	54	-
Qwen3-14B + Rose-SQL 2026.05	66.45	79.1	74.44	50.25	63.82	58.04
Qwen3-8B + Rose-SQL 2026.05	65.75	78.72	72.73	50	62.56	55.21
SFT Mistral 7B + Track-SQL 2026.03	65.41	73.23	67.83	46.91	54.73	48.57
Track-SQL 2026.05	65.41	75.39	69.16	46.91	57.81	50.71
SFT Deepseek 7B + Track-SQL 2026.03	65.17	75.39	69.16	46.44	57.81	50.71
HIE-SQL + GraPPA 2026.03	64.7	-	-	45	-	-
HIE-SQL + GraPPa 2026.05	64.7	-	-	45	-	-
SFT Deepseek 7B 2026.03	64.33	71.4	65.08	43.36	50.71	43.36
SFT Mistral 7B 2026.03	64.17	70.82	65.58	43.6	52.13	45.49
Qwen3-4B + Rose-SQL 2026.05	61.51	74.23	68.74	43.83	55.98	48.95
QDA-SQL 2026.03	61.3	-	-	44.1	-	-
QDA-SQL 2026.05	61.3	-	-	44.1	-	-
SFT Codellama 7B + Track-SQL 2026.03	61.26	70.07	63.75	43.36	52.13	45.73
gpt-oss-20b 2026.05	60.1	73.5	66.8	33.2	50.8	36.5
SFT Codellama 7B 2026.03	59.18	67.99	61.51	36.96	46.68	39.33
CoE-SQL 2026.03	56	70.3	63.3	36.5	50.5	41.9
CoE-SQL 2026.05	56	70.3	63.3	36.5	50.5	41.9
Qwen3-14B 2026.05	55.19	71.98	65.25	29.38	49.05	40.52
Qwen3-4B 2026.05	54.11	66	58.27	28.67	40.28	31.04
Qwen3-8B 2026.05	53.12	66.99	61.68	26.78	41.94	35.31
ACT-SQL 2026.03	51	63.8	56.9	24.4	38.9	29.6
ACT-SQL 2026.05	51	63.8	56.9	24.4	38.9	29.6
DeepSeek-R1-Distill-Llama-8B + Rose-SQL 2026.05	42.5	61.25	54.3	18.5	32.8	25.4
DeepSeek-R1-Distill-Llama-8B 2026.05	31.58	49.13	43.89	10.43	21.09	16.82