Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DeepEye-SQL: A Software-Engineering-Inspired Text-to-SQL Framework

About

Large language models (LLMs) have advanced Text-to-SQL, yet existing solutions still fall short of system-level reliability. The limitation is not merely in individual modules -- e.g., schema linking, reasoning, and verification -- but more critically in the lack of structured orchestration that enforces correctness across the entire workflow. This gap motivates a paradigm shift: treating Text-to-SQL not as free-form language generation but as a software-engineering problem that demands structured, verifiable orchestration. We present DeepEye-SQL, a software-engineering-inspired framework that reframes Text-to-SQL as the development of a small software program, executed through a verifiable process guided by the Software Development Life Cycle (SDLC). DeepEye-SQL integrates four synergistic stages: it grounds user intent through robust schema linking, enforcing relational closure; enhances fault tolerance with N-version SQL generation; ensures deterministic verification via a ``Syntax-Logic-Quality'' tool-chain that intercepts errors pre-execution; and introduces confidence-aware selection that leverages execution-guided adjudication to resolve ambiguity beyond simple majority voting. Leveraging open-source MoE LLMs (~30B total, ~3B activated parameters) without any fine-tuning, DeepEye-SQL achieves 73.5% execution accuracy on BIRD-Dev, 75.07% on the official BIRD-Test leaderboard, and 89.8% on Spider-Test, outperforming state-of-the-art solutions that rely on larger models or extensive training. This highlights that principled orchestration, rather than LLM scaling alone, is key to achieving system-level reliability in Text-to-SQL.

Boyan Li, Chong Chen, Zhujun Xue, Yinan Mei, Yuyu Luo• 2025

Related benchmarks

TaskDatasetResultRank
Text-to-SQLBIRD (dev)
Execution Accuracy (EA)73.53
477
Text-to-SQLSpider (dev)
EX84.44
196
Text-to-SQLSpider 1.0 (test)
EM Acc (Overall)89.8
110
Text-to-SQLSpider Lite 2.0
Execution Accuracy (EX)33.6
46
Text-to-SQLArcher (dev)
Execution Accuracy49
45
NL2SQLArcher Chinese zh (dev)
Execution Accuracy (EX)50
9
NL2SQLBEAVER Oracle dialect
Execution Accuracy14.1
9
NL2SQLAmbrosia Ambiguous
Execution Accuracy (EX)64.7
3
NL2SQLAMBROSIA Unambiguous
Execution Accuracy (EX)72
3
Showing 9 of 9 rows

Other info

Follow for update