BIRD
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Text-to-SQL | BIRD (dev) | Execution Accuracy (EA)74.46 | 217 | |
| Text2SQL | BIRD (dev) | Exec Acc (Greedy)65.3 | 37 | |
| Text-to-SQL | BIRD (test) | EX75.63 | 32 | |
| Image Classification | Bird | Accuracy85.1 | 29 | |
| SQL Semantic Validation | BIRD | AUPRC80.36 | 24 | |
| Text-to-SQL | BIRD | Total Execution Accuracy68.32 | 22 | |
| SQL execution performance | BIRD n=1534 | EM (1T)64.9 | 21 | |
| SQL Generation | BIRD Original (dev) | Execution Accuracy (Simple)65.51 | 14 | |
| SQL Generation | BIRD Verified | Execution Accuracy (Simple)69.41 | 14 | |
| Text-to-SQL | BIRD (test dev) | Execution Accuracy (EX)48.92 | 14 | |
| Table Selection | BIRD 2023 (test) | Avg #tables5.3 | 12 | |
| Text-to-SQL | BIRD (holdout test) | Execution Accuracy73 | 11 | |
| Image Super-Resolution | Bird | PSNR25.2998 | 7 | |
| End-to-end Question Answering | Bird | Accuracy20.6 | 6 | |
| Object Detection | Bird | Accuracy94.9 | 5 | |
| DB Routing | BIRD Route | R@179.62 | 5 | |
| Coding Agent | Bird | Pass@143.83 | 5 | |
| Retrieval | Bird | Precision42.7 | 3 | |
| Classification | BIRD | Accuracy72 | 3 | |
| Text-to-SQL | BIRD official (test) | Total Accuracy73.67 | 2 | |
| Gram matrix computation | Bird | Metric- | 0 |
Showing 21 of 21 rows