Code Generation on HumanEval and MBPP EvalPlus
70.1HumanEval+ Pass@k
Mistral-7B-Instruct-v0.2
Updated 3d ago
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
2024.05 | 70.1 | 75 | 44.7 | 37 | 59.9 | 53.6 | |
2024.05 | 65.9 | 72.1 | 60.1 | 50.4 | 66.1 | 58.2 | |
2025.03 | 62.8 | 66.5 | 63.4 | 51.4 | 61 | - | |
2025.03 | 62.8 | 65.9 | 62.9 | 51.9 | 60.9 | - | |
2025.03 | 61 | 64.6 | 63.9 | 51.4 | 60.2 | - | |
2024.05 | 57.9 | 63.4 | 60.4 | 48.6 | 61.9 | 53.3 | |
2024.05 | 56.7 | 61.6 | 70.1 | 59.3 | 65.9 | 58 | |
2024.05 | 53.7 | 57.9 | 68.7 | 56.9 | 63.3 | 55.3 | |
2025.02 | 51.83 | - | - | 48.68 | - | - | |
2024.05 | 39.6 | 45.1 | 59.5 | 49.7 | 52.3 | 44.7 | |
2025.02 | 39.02 | - | - | 39.42 | - | - | |
2024.05 | 35.4 | 42.7 | 57.1 | 45 | 49.9 | 40.2 | |
2025.02 | 34.76 | - | - | 50.26 | - | - | |
2024.05 | 29.3 | 33.5 | 61.4 | 51.6 | 47.5 | 40.5 | |
2025.02 | 24.39 | - | - | 27.51 | - | - | |
2024.05 | 23.8 | 28.7 | 51.9 | 42.1 | 40.3 | 33 | |
2025.02 | 23.78 | - | - | 21.69 | - | - | |
2025.02 | 21.34 | - | - | 21.34 | - | - | |
2025.02 | 20.73 | - | - | 22.75 | - | - | |
2025.02 | 20.4 | - | - | 17.7 | - | - | |
2025.02 | 20.12 | - | - | 22.22 | - | - | |
2024.05 | 20.1 | 26.8 | 52.6 | 43.4 | 39.7 | 31.8 | |
2025.02 | 18.9 | - | - | 23 | - | - | |
2025.02 | 18.9 | - | - | 21.42 | - | - | |
2025.02 | 18.9 | - | - | 23.81 | - | - | |
2025.02 | 17.1 | - | - | 22.2 | - | - | |
2025.02 | 17.07 | - | - | 23.38 | - | - | |
2025.02 | 17.07 | - | - | 22.49 | - | - | |
2025.02 | 15.85 | - | - | 22.22 | - | - |