MultiPL-E
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code Generation | MultiPL-E | Average Score76.5 | 35 | |
| Coding | MultiPL-E | Score74.87 | 20 | |
| Code Generation | MultiPL-E | Average Pass@179.5 | 19 | |
| Code Generation | MultiPL-E HumanEval translated from Python | C++ Pass Rate54.6 | 17 | |
| Multilingual Code Completion | Multipl-E | Pass@131.14 | 12 | |
| Code Generation | MultiPL-E 2022 (test) | Java44.9 | 10 | |
| Code Generation | MultiPL-E MBPP | Score58.8 | 9 | |
| Code Generation | MultiPL-E Java | Pass@142.07 | 6 | |
| Code Generation | MultiPL-E | Pass@1 (Lua)42 | 6 | |
| Code Synthesis | MultiPL-E | Success Rate (Lua)68 | 5 | |
| Code Generation | MultiPL-E | Accuracy59.6 | 5 | |
| Single line code infilling | MultiPL-E | Python SPM Exact Match74.5 | 5 | |
| Code Generation | MultiPL-E v1 (test) | Accuracy59.1 | 3 |
Showing 13 of 13 rows