Code Generation on HumanEval 0-shot (test)
45.7Accuracy
Prefix-Cache
Updated 2d ago
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
2026.01 | 45.7 | - | - | 21.4 | 1.8 | - | - | - | - | - | |
2025.12 | 45.12 | - | - | - | - | 94.41 | 1 | - | - | - | |
2026.01 | 45.1 | - | - | 32.9 | 1.2 | - | - | - | - | - | |
2026.01 | 44.5 | - | - | 9 | 4.4 | - | - | - | - | - | |
2026.01 | 44.5 | - | - | 5.2 | 7.6 | - | - | - | - | - | |
2026.01 | 43.9 | - | - | 39.4 | 1 | - | - | - | - | - | |
2026.05 | 43.7 | - | - | - | 3.8 | 24.7 | - | - | - | - | |
2026.01 | 43.4 | - | - | 2.5 | 5.3 | - | - | - | - | - | |
2026.01 | 43.3 | - | - | 3.4 | 3.9 | - | - | - | - | - | |
2026.05 | 43.3 | - | - | - | 3.7 | 24.1 | - | - | - | - | |
2025.12 | 43.29 | - | - | - | - | 163.72 | 1.74 | - | - | - | |
2026.05 | 43 | - | - | - | 8.2 | 53.3 | - | - | - | - | |
2026.01 | 42.7 | - | - | 7.9 | 1.7 | - | - | - | - | - | |
2026.05 | 42.5 | - | - | - | 4.2 | 27.3 | - | - | - | - | |
2026.01 | 41.5 | - | - | 13.3 | 1 | - | - | - | - | - | |
2026.05 | 41.5 | - | - | - | 1 | 6.5 | - | - | - | - | |
2026.05 | 41.5 | - | - | - | 1.2 | 7.8 | - | - | - | - | |
2025.12 | 41.46 | - | - | - | - | 131.14 | 1.38 | - | - | - | |
2026.05 | 40.3 | - | - | - | 3.6 | 23.4 | - | - | - | - | |
2026.01 | 39.6 | - | - | 12.9 | 1 | - | - | - | - | - | |
2025.12 | 37.19 | - | - | - | - | 96.84 | 1 | - | - | - | |
2025.12 | 35.97 | - | - | - | - | 220.81 | 2.28 | - | - | - | |
2025.12 | 32.92 | - | - | - | - | 132.28 | 1.37 | - | - | - | |
2025.12 | - | 26.8 | - | - | - | - | - | - | - | - | |
2025.12 | - | 37.8 | - | - | - | - | - | - | - | - | |
2025.12 | - | 48.8 | - | - | - | - | - | - | - | - | |
2025.12 | - | - | 37.8 | - | - | - | - | - | - | - | |
2025.12 | - | 25.6 | - | - | - | - | - | - | - | - | |
2025.12 | - | 36 | - | - | - | - | - | - | - | - | |
2025.12 | - | 50 | - | - | - | - | - | - | - | - | |
2025.12 | - | - | 37.2 | - | - | - | - | - | - | - | |
2025.12 | - | 34.7 | - | - | - | - | - | - | - | - | |
2025.12 | - | 38.4 | - | - | - | - | - | - | - | - | |
2025.12 | - | 38.4 | - | - | - | - | - | - | - | - | |
2025.12 | - | - | 37.2 | - | - | - | - | - | - | - | |
2025.12 | - | 28.1 | - | - | - | - | - | - | - | - | |
2025.12 | - | 42.1 | - | - | - | - | - | - | - | - | |
2025.12 | - | 50 | - | - | - | - | - | - | - | - | |
2025.12 | - | - | 40.1 | - | - | - | - | - | - | - | |
2025.12 | - | 29.3 | - | - | - | - | - | - | - | - | |
2025.12 | - | 39.6 | - | - | - | - | - | - | - | - | |
2025.12 | - | 51.9 | - | - | - | - | - | - | - | - | |
2025.12 | - | - | 40.3 | - | - | - | - | - | - | - | |
2025.09 | - | 46.6 | - | 11.89 | 9.7 | 48.39 | - | - | - | - | |
2025.09 | - | 57.3 | - | 6.3 | 23.8 | 104.4 | - | - | - | - | |
2025.11 | - | - | - | 11.3 | - | 7.4 | - | 256 | 83.9 | 37.8 | |
2025.11 | - | - | - | 10.8 | - | 12.6 | - | 256 | 135.9 | 39 | |
2025.11 | - | - | - | 4.6 | - | 17.9 | - | 100.3 | 83.1 | 36.6 | |
2025.11 | - | - | - | 4.2 | - | 18.9 | - | 97.7 | 80.2 | 36 | |
2025.11 | - | - | - | 1.9 | - | 50.9 | - | 32.3 | 95.1 | 40.2 | |
2026.03 | - | 32.32 | - | - | - | 23.84 | - | - | - | - | |
2026.03 | - | 32.32 | - | - | - | 177.4 | - | - | - | - | |
2026.03 | - | 31.71 | - | - | - | 232.66 | - | - | - | - |