BIRD
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Text-to-SQL | BIRD (dev) | Execution Accuracy (EA)74.46 | 251 | |
| Text-to-SQL | BIRD | Total Execution Accuracy70.5 | 64 | |
| Text-to-SQL | BIRD (test) | EX75.63 | 46 | |
| Text2SQL | BIRD (dev) | Exec Acc (Greedy)65.8 | 44 | |
| Image Classification | Bird | Accuracy85.1 | 29 | |
| Color Video Completion | Bird color video sequence | PSNR25.382 | 28 | |
| Text-to-SQL | BIRD | Accuracy69.1 | 27 | |
| SQL Semantic Validation | BIRD | AUPRC80.36 | 24 | |
| SQL execution performance | BIRD n=1534 | EM (1T)64.9 | 21 | |
| Schema retrieval | BIRD | Recall (R)100 | 15 | |
| SQL Generation | BIRD Original (dev) | Execution Accuracy (Simple)65.51 | 14 | |
| SQL Generation | BIRD Verified | Execution Accuracy (Simple)69.41 | 14 | |
| Text-to-SQL | BIRD (test dev) | Execution Accuracy (EX)48.92 | 14 | |
| Table Selection | BIRD 2023 (test) | Avg #tables5.3 | 12 | |
| Text-to-SQL | BIRD (holdout test) | Execution Accuracy73 | 11 | |
| Text2SQL | BIRD Movies | Execution Accuracy46.9 | 9 | |
| Text2SQL | BIRD App Store | Execution Accuracy38.4 | 9 | |
| Text2SQL | BIRD Computer Students | Execution Accuracy48.3 | 9 | |
| Table Retrieval | BIRD | Capped Recall@2599.1 | 9 | |
| SQL generation | Bird | Greedy70.5 | 9 | |
| Retrieval | Bird | Recall@1096.1 | 9 | |
| Text-to-SQL | BIRD Movies | Spearman Correlation0.65 | 8 | |
| Text-to-SQL | BIRD Apps | Spearman Correlation0.74 | 8 | |
| Text-to-SQL | BIRD Computer | Spearman Correlation0.46 | 8 | |
| Table Retrieval | Bird (dev) | Precision@586.2 | 8 |
Showing 25 of 44 rows