Code Generation on HumanEval 0-shot (test)
57.3Pass@1
d2Cache
Updated 4d ago
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
2025.09 | 57.3 | - | - | 6.3 | 23.8 | 104.4 | - | - | - | - | |
2025.12 | 51.9 | - | - | - | - | - | - | - | - | - | |
2025.12 | 50 | - | - | - | - | - | - | - | - | - | |
2025.12 | 50 | - | - | - | - | - | - | - | - | - | |
2025.12 | 48.8 | - | - | - | - | - | - | - | - | - | |
2025.09 | 46.6 | - | - | 11.89 | 9.7 | 48.39 | - | - | - | - | |
2025.12 | 42.1 | - | - | - | - | - | - | - | - | - | |
2025.12 | 39.6 | - | - | - | - | - | - | - | - | - | |
2025.12 | 38.4 | - | - | - | - | - | - | - | - | - | |
2025.12 | 38.4 | - | - | - | - | - | - | - | - | - | |
2025.12 | 37.8 | - | - | - | - | - | - | - | - | - | |
2025.12 | 36 | - | - | - | - | - | - | - | - | - | |
2025.12 | 34.7 | - | - | - | - | - | - | - | - | - | |
2025.12 | 29.3 | - | - | - | - | - | - | - | - | - | |
2025.12 | 28.1 | - | - | - | - | - | - | - | - | - | |
2025.12 | 26.8 | - | - | - | - | - | - | - | - | - | |
2025.12 | 25.6 | - | - | - | - | - | - | - | - | - | |
2025.12 | - | 37.8 | - | - | - | - | - | - | - | - | |
2025.12 | - | 37.2 | - | - | - | - | - | - | - | - | |
2025.12 | - | 37.2 | - | - | - | - | - | - | - | - | |
2025.12 | - | 40.1 | - | - | - | - | - | - | - | - | |
2025.12 | - | 40.3 | - | - | - | - | - | - | - | - | |
2026.01 | - | - | 41.5 | 13.3 | 1 | - | - | - | - | - | |
2026.01 | - | - | 39.6 | 12.9 | 1 | - | - | - | - | - | |
2026.01 | - | - | 42.7 | 7.9 | 1.7 | - | - | - | - | - | |
2026.01 | - | - | 43.3 | 3.4 | 3.9 | - | - | - | - | - | |
2026.01 | - | - | 43.4 | 2.5 | 5.3 | - | - | - | - | - | |
2026.01 | - | - | 43.9 | 39.4 | 1 | - | - | - | - | - | |
2026.01 | - | - | 45.1 | 32.9 | 1.2 | - | - | - | - | - | |
2026.01 | - | - | 45.7 | 21.4 | 1.8 | - | - | - | - | - | |
2026.01 | - | - | 44.5 | 9 | 4.4 | - | - | - | - | - | |
2026.01 | - | - | 44.5 | 5.2 | 7.6 | - | - | - | - | - | |
2025.12 | - | - | 35.97 | - | - | 220.81 | 2.28 | - | - | - | |
2025.12 | - | - | 43.29 | - | - | 163.72 | 1.74 | - | - | - | |
2025.12 | - | - | 32.92 | - | - | 132.28 | 1.37 | - | - | - | |
2025.12 | - | - | 41.46 | - | - | 131.14 | 1.38 | - | - | - | |
2025.12 | - | - | 37.19 | - | - | 96.84 | 1 | - | - | - | |
2025.12 | - | - | 45.12 | - | - | 94.41 | 1 | - | - | - | |
2025.11 | - | - | - | 11.3 | - | 7.4 | - | 256 | 83.9 | 37.8 | |
2025.11 | - | - | - | 10.8 | - | 12.6 | - | 256 | 135.9 | 39 | |
2025.11 | - | - | - | 4.6 | - | 17.9 | - | 100.3 | 83.1 | 36.6 | |
2025.11 | - | - | - | 4.2 | - | 18.9 | - | 97.7 | 80.2 | 36 | |
2025.11 | - | - | - | 1.9 | - | 50.9 | - | 32.3 | 95.1 | 40.2 |