CodeMMLU
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Understanding | CodeMMLU | Exact Match (EM)53.52 | 28 | |
| Code Evaluation | CodeMMLU software principles | Accuracy24.9 | 12 | |
| Coding Reasoning | CodeMMLU | Pass@482.8 | 6 |
Showing 3 of 3 rows
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Understanding | CodeMMLU | Exact Match (EM)53.52 | 28 | |
| Code Evaluation | CodeMMLU software principles | Accuracy24.9 | 12 | |
| Coding Reasoning | CodeMMLU | Pass@482.8 | 6 |