LeetCode
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Generation | LeetCode Contest Benchmark | Easy Accuracy89.7 | 36 | |
| Code Generation | LeetCode without write access | Pass@115.2 | 16 | |
| Reward Hacking | LeetCode with write access (test) | Hack Rate1 | 16 | |
| Code Generation | Leetcode-New | Pass Rate49.08 | 12 | |
| Code Generation | LeetCodeHard | Accuracy38.2 | 8 | |
| Unit test generation | LeetCode (test) | Error Rate (%)2.77 | 7 | |
| Code Generation | Leetcode-Guided | Pass Rate48.02 | 6 | |
| Code Generation | Leetcode Original | Pass Rate38.6 | 6 | |
| Code Generation | LeetCode (OOD) | Accuracy27.8 | 5 | |
| Code Generation | LeetCode Weekly Contests 484-488 | Pass Rate85 | 4 | |
| Software Engineering / Coding | Leetcode hard | Hard LeetCode Accuracy26 | 1 | |
| Software Engineering / Coding | Leetcode medium | Metric- | 0 | |
| Software Engineering / Coding | Leetcode easy | Metric- | 0 |
Showing 13 of 13 rows