Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration

About

We present ReFoRCE, a Text-to-SQL agent that tops the Spider 2.0 leaderboard--a challenging benchmark reflecting complex, real-world Text-to-SQL scenarios. While Text-to-SQL systems enable natural language queries over structured databases, deploying them in enterprise environments remains difficult due to large, complex schemas (with over 1,000 columns), diverse SQL dialects (e.g., BigQuery, Snowflake), and sophisticated query requirements (e.g., transformations and analytics). ReFoRCE addresses these challenges through: (a) database information compression via pattern-based table grouping and LLM-guided schema linking to alleviate long-context issues; (b) self-refinement to iteratively correct syntax and semantic errors across dialects; (c) majority-vote consensus to select high-confidence candidates while deferring ambiguous cases arising from sophisticated queries; and (d) iterative column exploration guided by execution feedback to resolve those deferred cases. ReFoRCE achieves new state-of-the-art results, with scores of 35.83 on Spider 2.0-Snow and 36.56 on Spider 2.0-Lite.

Minghang Deng, Ashwin Ramachandran, Canwen Xu, Lanxiang Hu, Zhewei Yao, Anupam Datta, Hao Zhang• 2025

Related benchmarks

TaskDatasetResultRank
Text-to-SQLEHRSQL
Execution Accuracy31.2
69
Text-to-SQLSpider Lite 2.0
Execution Accuracy (EX)36.56
46
Knowledge-intensive Claim Extraction and Information Retrieval75-case (test)
Overall Score0.62
35
Text-to-SQLSpider Snow 2.0 (test)
Execution Accuracy (EX)36.6
21
Schema linkingBIRD (dev)
SRR39.63
16
Text-to-SQLSpider2-Snow 80-case
Execution Accuracy36.25
9
Database Question AnsweringRealEHR (test)
Accuracy48.6
8
Question AnsweringRealEHR
Accuracy48.6
8
Database Question AnsweringLS-Hard (test)
Answer Accuracy40.1
8
Question AnsweringLS-Hard
Accuracy40.1
8
Showing 10 of 20 rows

Other info

Follow for update