PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents
About
Text-to-SQL systems often struggle with deep contextual understanding, particularly for complex queries with subtle requirements. We present PV-SQL, an agentic framework that addresses these failures through two complementary components: Probe and Verify. The Probe component iteratively generates probing queries to retrieve concrete records from the database, resolving ambiguities in value formats, column semantics, and inter-table relationships to build richer contextual understanding. The Verify component employs a rule-based method to extract verifiable conditions and construct an executable checklist, enabling iterative SQL refinement that effectively reduces missing constraints. Experiments on the BIRD benchmarks show that PV-SQL outperforms the best text-to-SQL baseline by 5% in execution accuracy and 20.8% in valid efficiency score while consuming fewer tokens.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Text-to-SQL | BIRD (dev) | Execution Accuracy (EA)63.62 | 387 | |
| Text-to-SQL | Spider (dev) | EX75.55 | 147 | |
| Text-to-SQL | Spider | Exec Acc (All)77.66 | 139 | |
| Text-to-SQL | Bird | Execution Accuracy (EX)65.12 | 63 | |
| Text-to-SQL | BIRD-SQL Mini (dev) | Execution Accuracy (EX)63.8 | 21 | |
| Text-to-SQL | Bird | Execution Accuracy65.1 | 20 | |
| Text-to-SQL | Mini (dev) | Execution Accuracy (EX)63.8 | 9 |