Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruning
About
We focus on Text-to-SQL semantic parsing from the perspective of retrieval-augmented generation. Motivated by challenges related to the size of commercial database schemata and the deployability of business intelligence solutions, we propose $\text{ASTReS}$ that dynamically retrieves input database information and uses abstract syntax trees to select few-shot examples for in-context learning. Furthermore, we investigate the extent to which an in-parallel semantic parser can be leveraged for generating approximated versions of the expected SQL queries, to support our retrieval. We take this approach to the extreme--we adapt a model consisting of less than $500$M parameters, to act as an extremely efficient approximator, enhancing it with the ability to process schemata in a parallelised manner. We apply $\text{ASTReS}$ to monolingual and cross-lingual benchmarks for semantic parsing, showing improvements over state-of-the-art baselines. Comprehensive experiments highlight the contribution of modules involved in this retrieval-augmented generation setting, revealing interesting directions for future work.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Text-to-SQL | Spider (dev) | EX (All)86.6 | 100 | |
| Text-to-SQL | Spider-Realistic | Execution Accuracy (EX)80.9 | 33 | |
| Text-to-SQL | Spider-Syn | Execution Accuracy (EX)74.4 | 26 | |
| Text-to-SQL | Spider-DK | Execution Accuracy (EX)72.3 | 26 |