CodeContests
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Generation | CodeContests (test) | Pass@11,200 | 48 | |
| Code Generation | CodeContests | Pass@158.79 | 38 | |
| Code Generation | CodeContests | Avg@839.33 | 26 | |
| Code Generation | CodeContests | Accuracy (CC)26.7 | 15 | |
| Code Generation | CodeContests (evaluation set) | Pass@119.7 | 8 | |
| Program Synthesis | CodeContests (test) | Pass@10.2045 | 6 | |
| Coding Reasoning | Codecontests | Pass Rate65.8 | 5 | |
| Competition-Level Code Generation | CodeContests (val) | 10@1k21 | 3 |
Showing 8 of 8 rows