BIRD
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Text-to-SQL | BIRD (dev) | Execution Accuracy (EA)77.84 | 387 | |
| Text-to-SQL | BIRD | Total Execution Accuracy97.59 | 68 | |
| Text-to-SQL | BIRD | Execution Accuracy (EX)65.12 | 63 | |
| Text-to-SQL | BIRD (test) | EX75.63 | 46 | |
| Text2SQL | BIRD (dev) | Exec Acc (Greedy)65.8 | 44 | |
| Text-to-SQL | BIRD (Non-Synthesized Matched Set) | ExM Accuracy93.15 | 32 | |
| Text-to-SQL | BIRD Synthesized Matched Set | ExM Accuracy90.39 | 32 | |
| Image Classification | Bird | Accuracy85.1 | 29 | |
| Color Video Completion | Bird color video sequence | PSNR25.382 | 28 | |
| Text-to-SQL | BIRD | Accuracy69.1 | 27 | |
| SQL Semantic Validation | BIRD | AUPRC80.36 | 24 | |
| Text-to-SQL | BIRD | Execution Accuracy (Llama-8B)31.8 | 21 | |
| SQL execution performance | BIRD n=1534 | EM (1T)64.9 | 21 | |
| Table Retrieval | BIRD union (test) | Precision54.4 | 20 | |
| Text-to-SQL | BIRD | Execution Accuracy73 | 20 | |
| Schema linking | BIRD (dev) | SRR100 | 16 | |
| Text-to-SQL | Bird | Match Accuracy (MAT)6.67 | 15 | |
| Schema retrieval | BIRD | Recall (R)100 | 15 | |
| SQL Generation | BIRD Original (dev) | Execution Accuracy (Simple)65.51 | 14 | |
| SQL Generation | BIRD Verified | Execution Accuracy (Simple)69.41 | 14 | |
| Text-to-SQL | BIRD (test dev) | Execution Accuracy (EX)48.92 | 14 | |
| Text-to-SQL | BIRD | Kendall's τ-0.11 | 12 | |
| Table Selection | BIRD 2023 (test) | Avg #tables5.3 | 12 | |
| Table Retrieval | BIRD | Precision (P)57.3 | 11 | |
| SQL generation | Bird | Pass@160.6 | 11 |
Showing 25 of 62 rows