Code Generation on HumanEval Multilingual (test)
76.5Average Score
GPT-4
Updated 4d ago
Evaluation Results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2024.01 | 76.5 | 84.1 | 76.4 | 81.6 | 77.2 | 77.4 | 79.1 | 58.2 | 78 | - | - | - | - | - | |
2024.01 | 69.2 | 79.3 | 68.9 | 73.4 | 72.7 | 67.9 | 74.1 | 43 | 73.9 | - | - | - | - | - | |
2024.01 | 66.1 | 78.6 | 63.4 | 68.4 | 68.9 | 67.2 | 72.8 | 36.7 | 72.7 | - | - | - | - | - | |
2024.01 | 64.9 | 76.2 | 63.4 | 69.2 | 60.9 | 69.1 | 70.8 | 42.4 | 67.1 | - | - | - | - | - | |
2024.01 | 50.3 | 56.1 | 58.4 | 51.9 | 44.1 | 52.8 | 51.3 | 32.3 | 55.3 | - | - | - | - | - | |
2024.01 | 48.4 | 65.2 | 45.3 | 51.9 | 45.3 | 59.7 | 55.1 | 12.7 | 52.2 | - | - | - | - | - | |
2024.02 | 46.8 | - | - | - | - | - | - | - | - | 60.4 | 43.9 | 40.9 | 49.4 | 39.6 | |
2023.08 | 45.9 | - | 53.4 | 58.2 | 58.4 | 39 | 36.7 | 2,970 | - | - | - | - | - | - | |
2023.08 | 45.3 | - | 52.8 | 51.9 | 50.9 | 49.1 | 38 | 2,910 | - | - | - | - | - | - | |
2023.08 | 45 | - | 54.7 | 57.6 | 53.4 | 44 | 34.8 | 2,530 | - | - | - | - | - | - | |
2024.01 | 44.7 | 49.4 | 50.3 | 43 | 38.5 | 49.7 | 50 | 28.5 | 48.4 | - | - | - | - | - | |
2024.01 | 41 | 48.2 | 44.7 | 44.9 | 41 | 42.1 | 48.7 | 15.8 | 42.2 | - | - | - | - | - | |
2023.08 | 36.4 | - | 47.8 | 45.6 | 44.1 | 33.3 | 30.4 | 1,710 | - | - | - | - | - | - | |
2023.08 | 36.1 | - | 45.3 | 43.7 | 36.6 | 40.3 | 31 | 1,960 | - | - | - | - | - | - | |
2024.01 | 35.4 | 36 | 37.9 | 38 | 34.2 | 45.2 | 43 | 16.5 | 32.3 | - | - | - | - | - | |
2023.08 | 35.1 | - | 42.2 | 44.9 | 42.9 | 34.3 | 31.7 | 1,460 | - | - | - | - | - | - | |
2025.12 | 34.4 | - | - | - | - | - | - | - | - | - | 48.2 | 31.1 | 23.8 | - | |
2025.12 | 34.2 | - | - | - | - | - | - | - | - | - | 36.6 | 37.2 | 28.7 | - | |
2025.12 | 33.3 | - | - | - | - | - | - | - | - | - | 43.9 | 27.4 | 28.7 | - | |
2023.08 | 32 | - | 42.2 | 40.5 | 32.3 | 39 | 24 | 1,390 | - | - | - | - | - | - | |
2023.08 | 31.5 | - | 39.1 | 37.3 | 33.5 | 35.2 | 29.8 | 1,390 | - | - | - | - | - | - | |
2025.12 | 31.5 | - | - | - | - | - | - | - | - | - | 48.2 | 22.6 | 23.8 | - | |
2023.08 | 30.6 | - | 39.1 | 38 | 34.2 | 29.6 | 27.3 | 1,520 | - | - | - | - | - | - | |
2024.01 | 29.2 | 31.7 | 29.8 | 34.2 | 23.6 | 36.5 | 36.7 | 12 | 29.2 | - | - | - | - | - | |
2024.01 | 28.3 | 34.8 | 31.1 | 32.3 | 24.2 | 28.9 | 36.7 | 10.1 | 28.6 | - | - | - | - | - | |
2024.01 | 28 | 31.7 | 31.1 | 28.5 | 25.4 | 34 | 34.8 | 8.9 | 29.8 | - | - | - | - | - | |
2023.08 | 27.5 | - | 32.3 | 35.4 | 32.4 | 23.9 | 24.7 | 1,650 | - | - | - | - | - | - | |
2023.08 | 26.3 | - | 28.6 | 34.2 | 24.2 | 33.3 | 25.3 | 1,200 | - | - | - | - | - | - | |
2023.08 | 26.1 | - | 30.6 | 31.9 | 28.9 | 31.3 | 22.1 | 1,170 | - | - | - | - | - | - | |
2023.08 | 25.8 | - | 31.1 | 30.4 | 28.6 | 32.7 | 21.6 | 1,010 | - | - | - | - | - | - | |
2023.08 | 25.3 | - | 31.6 | 30.2 | 26.1 | 32.3 | 21 | 1,050 | - | - | - | - | - | - | |
2023.08 | 25 | - | 30.6 | 28.5 | 26.8 | 32.2 | 20.6 | 1,100 | - | - | - | - | - | - | |
2024.01 | 24.5 | 36 | 29.2 | 25.9 | 23.6 | 20.8 | 29.7 | 6.3 | 24.8 | - | - | - | - | - | |
2023.08 | 24.4 | - | 30.4 | 31.7 | 34.2 | 15.1 | 25.9 | 890 | - | - | - | - | - | - | |
2025.12 | 22.6 | - | - | - | - | - | - | - | - | - | 20.7 | 23.2 | 23.8 | - | |
2025.12 | 20.3 | - | - | - | - | - | - | - | - | - | 19.5 | 20.1 | 21.3 | - | |
2025.12 | 18.8 | - | - | - | - | - | - | - | - | - | 21.7 | 18.3 | 16.5 | - | |
2023.08 | 18 | - | 23.6 | 22.2 | 19.9 | 21.4 | 17.1 | 380 | - | - | - | - | - | - | |
2025.12 | 17.7 | - | - | - | - | - | - | - | - | - | 17.1 | 18.3 | 17.7 | - | |
2025.12 | 17.7 | - | - | - | - | - | - | - | - | - | 20.9 | 15.6 | 16.5 | - | |
2024.02 | 13.9 | - | - | - | - | - | - | - | - | 14.6 | 13.4 | 14.6 | 12.8 | 14 | |
2023.08 | 13.4 | - | 21 | 22.2 | 8.4 | 20.1 | 8.2 | 60 | - | - | - | - | - | - | |
2024.02 | 12.2 | - | - | - | - | - | - | - | - | 11 | 10.4 | 8.5 | 1.2 | 13.4 | |
2024.02 | 12.2 | - | - | - | - | - | - | - | - | 18.3 | 15.9 | 9.8 | 8.5 | 14.6 | |
2023.08 | 11.8 | - | 16.9 | 19.1 | 13.5 | 10.1 | 8.5 | 280 | - | - | - | - | - | - | |
2023.08 | 11.4 | - | 13.7 | 15.8 | 13.1 | 13.2 | 9.5 | 320 | - | - | - | - | - | - | |
2024.02 | 9.9 | - | - | - | - | - | - | - | - | 9.2 | 6.7 | 11 | 9.8 | 12.8 | |
2024.02 | 9.8 | - | - | - | - | - | - | - | - | 12.2 | 11 | 8.5 | 8.5 | 8.5 | |
2023.08 | 8.3 | - | 6.8 | 10.8 | 9.9 | 12.6 | 6.3 | 320 | - | - | - | - | - | - | |
2024.02 | 8.1 | - | - | - | - | - | - | - | - | 10.4 | 7.9 | 6.1 | 7.3 | 8.5 | |
2024.02 | 7.3 | - | - | - | - | - | - | - | - | 8.5 | 7.3 | 6.1 | 8.5 | 6.1 | |
2024.02 | 6.8 | - | - | - | - | - | - | - | - | 8.5 | 7.3 | 6.1 | 6.1 | 6.1 | |
2024.01 | - | 33.5 | 31.9 | 30.6 | 28.9 | 31.3 | 22.1 | 11.7 | - | - | - | - | - | - |