SParC
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Dataset-level accuracy estimation | SParC to CoSQL | MAE1.5 | 54 | |
| Multi-turn Text-to-SQL | SParC In-domain | Exact Match71.5 | 29 | |
| Multi-turn Text-to-SQL | SparC (dev) | QM EM68.5 | 27 | |
| Context-dependent Text-to-SQL | SParC 1.0 (dev) | Question Match67.7 | 27 | |
| Multi-turn Text-to-SQL | SParC Out-of-domain | Execution Accuracy (EX)77.4 | 16 | |
| Text-to-SQL | SParC | Kendall's Tau-0.43 | 12 | |
| Context-dependent Text-to-SQL | SParC (test) | Question Match65.7 | 12 | |
| Context-dependent Text-to-SQL | SParC 1.0 (test) | Question Match67.7 | 12 | |
| Full Pipeline | SPARC | Speedup8.5 | 8 | |
| Articulated-Body Algorithm (ABA) | SPARC | Speedup9.2 | 8 | |
| Composite Rigid Body Algorithm (CRBA) | SPARC | Speedup7.9 | 8 | |
| Recursive Newton-Euler Algorithm (RNEA) | SPARC | Speedup7.1 | 8 | |
| Jacobian Computation (Jac.) | SPARC | Speedup19 | 8 | |
| Forward Kinematics (FK) | SPARC | Speedup25 | 8 | |
| Conversational text-to-SQL | SParC (dev) | Question Match64.1 | 7 | |
| Semantic Parsing | SParC (dev) | Query Accuracy42.4 | 6 | |
| Radial acceleration relation modeling | SPARC database 2017 | wMSE0.07 | 5 | |
| Rigid Body Dynamics | SPARC | FK17 | 4 | |
| Rigid Body Dynamics | SPARC 11 DOF | FK23 | 4 | |
| Text-To-SQL | SParC | Execution Accuracy85.78 | 4 | |
| Language-to-SQL Parsing | Sparc (test) | Exact Match (EM)47.9 | 3 | |
| Language-to-SQL Parsing | Sparc (dev) | EM48.9 | 3 | |
| Multi-turn Text-to-SQL | SParC (ADVETA-ADD) | EM42.9 | 2 | |
| Multi-turn Text-to-SQL | SParC (ADVETA-RPL) | Exact Match (EM)34.2 | 2 | |
| Multi-turn Text-to-SQL | SParC original (dev) | Exact Match (EM)50.7 | 2 |
Showing 25 of 25 rows