DS-1000
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Generation | DS-1000 | Pass@158.65 | 28 | |
| Code Generation | DS-1000 1.0 (test) | Matplotlib67.2 | 19 | |
| Data science code generation | DS-1000 | Matplotlib Score60.3 | 13 | |
| Code Generation | DS-1000 | Accuracy69.9 | 11 | |
| Data Science Code Completion | DS-1000 | Pandas (Pass@1)32 | 9 | |
| Data Science | DS-1000 | Performance Score56.5 | 8 | |
| Code Completion | DS-1000 (test) | Matplotlib Success Rate56.1 | 8 | |
| Code Generation | DS-1000 Python | Pass@139.7 | 7 | |
| Code Completion | DS-1000 1.0 (test) | Success Rate (Matplotlib)55.2 | 5 | |
| Code Insertion | DS-1000 1.0 (test) | Success Rate (Matplotlib)55.2 | 3 |
Showing 10 of 10 rows