MultiPL-E
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Code correctness prediction | MultiPL-E Java | AUROC0.705 | 60 | |
| Code Correctness Prediction | MultiPL-E Java | Brier Score0.231 | 60 | |
| Code Correctness Prediction | MultiPL-E Java | ECE0.075 | 60 | |
| Code Generation | MultiPL-E | Average Score76.5 | 47 | |
| Coding | MultiPL-E | Score87.9 | 31 | |
| Code Generation | MultiPL-E | Average Pass@179.5 | 19 | |
| Code Generation | MultiPL-E HumanEval translated from Python | C++ Pass Rate54.6 | 17 | |
| Multilingual Code Completion | Multipl-E | Pass@131.14 | 12 | |
| Multilingual Code Generation | Multipl-E | MultiplE72.84 | 10 | |
| Code Generation | MultiPL-E 2022 (test) | Java44.9 | 10 | |
| Code Generation | MultiPL-E MBPP | Score58.8 | 9 | |
| Code Generation | MultiPL-E Java | Pass@142.07 | 6 | |
| Code Generation | MultiPL-E | Pass@1 (Lua)42 | 6 | |
| Code Generation | MultiPL-E 7 langs | Score (%)26 | 5 | |
| Code Generation | MultiPL-E | Pass@161.1 | 5 | |
| Code Synthesis | MultiPL-E | Success Rate (Lua)68 | 5 | |
| Code Generation | MultiPL-E | Accuracy59.6 | 5 | |
| Single line code infilling | MultiPL-E | Python SPM Exact Match74.5 | 5 | |
| Code Generation | MultiPL-E v1 (test) | Accuracy59.1 | 3 |
Showing 19 of 19 rows