Code Generation on MBPP (accuracy, throughput, speedup)
90.5Accuracy
Claude 3.5 Sonnet
Updated 14d ago
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
2026.03 | 90.5 | - | - | |
2026.03 | 88.6 | - | - | |
2026.03 | 87.8 | - | - | |
2026.03 | 86 | - | - | |
2026.03 | 79 | - | - | |
2026.03 | 78.8 | - | - | |
2026.03 | 77.24 | - | - | |
2026.03 | 77.1 | - | - | |
2026.03 | 73.8 | - | - | |
2026.03 | 73.5 | - | - | |
2026.03 | 71.7 | - | - | |
2026.05 | 67.7 | 83 | - | |
2026.05 | 67.3 | 1.42 | - | |
2026.03 | 66.6 | - | - | |
2026.05 | 66.5 | 122.6 | - | |
2026.05 | 66.5 | 1.31 | - | |
2026.05 | 63.8 | 1.58 | - | |
2026.03 | 63 | - | - | |
2026.03 | 62.4 | - | - | |
2026.03 | 60.6 | - | - | |
2025.09 | 60 | - | - | |
2026.05 | 59.9 | 1.8 | - | |
2026.01 | 58.8 | 4.7 | 1 | |
2025.09 | 57 | - | - | |
2026.01 | 56.2 | 4.7 | 1 | |
2025.09 | 56 | - | - | |
2026.05 | 55.6 | 1.62 | - | |
2026.01 | 55.4 | 31.1 | 6.6 | |
2026.01 | 54.4 | 29.7 | 6.3 | |
2026.01 | 53.8 | 15.9 | 3.4 | |
2026.05 | 53.3 | 1.2 | - | |
2026.01 | 53.2 | 25.3 | 5.4 | |
2026.01 | 53.2 | 6.4 | 1.4 | |
2026.05 | 52.9 | 213.1 | - | |
2026.01 | 51 | 10.1 | 2.2 | |
2026.05 | 51 | 1.84 | - | |
2026.03 | 50.3 | - | - | |
2026.01 | 50.2 | 24.9 | 5.3 | |
2026.01 | 48.6 | 10 | 2.1 | |
2026.05 | 48.2 | 1.69 | - | |
2026.02 | 47.4 | - | - | |
2026.02 | 47.2 | - | - | |
2026.02 | 45.6 | - | - | |
2026.05 | 45.5 | 1.49 | - | |
2026.05 | 42.4 | 1.63 | - | |
2026.02 | 42 | - | - | |
2025.07 | 39.2 | - | - | |
2026.05 | 37.7 | 1.78 | - | |
2025.07 | 36.8 | - | - | |
2025.07 | 36.8 | - | - | |
2025.07 | 36.6 | - | - | |
2026.02 | 36.6 | - | - | |
2025.07 | 36.4 | - | - | |
2026.02 | 36.2 | - | - | |
2025.07 | 35.6 | - | - | |
2026.02 | 35.6 | - | - | |
2026.02 | 35.6 | - | - | |
2026.02 | 35.2 | - | - | |
2026.02 | 34.8 | - | - | |
2026.02 | 34 | - | - | |
2026.05 | 32.3 | 1.64 | - | |
2025.09 | 32.2 | - | - | |
2026.02 | 32.2 | - | - | |
2026.02 | 32.2 | - | - | |
2026.02 | 29.8 | - | - | |
2026.02 | 29.4 | - | - | |
2026.02 | 27.8 | - | - | |
2026.02 | 26.6 | - | - | |
2025.09 | 26.3 | - | - | |
2025.09 | 24.7 | - | - | |
2026.02 | 24.2 | - | - | |
2026.02 | 23.8 | - | - | |
2025.09 | 23.7 | - | - | |
2026.02 | 23.6 | - | - | |
2026.02 | 22.8 | - | - | |
2026.02 | 21.6 | - | - | |
2026.02 | 19.4 | - | - | |
2025.09 | 19.3 | - | - | |
2025.09 | 18.9 | - | - | |
2026.02 | 18 | - | - | |
2026.02 | 17 | - | - | |
2026.02 | 9.4 | - | - | |
2026.02 | 8.8 | - | - | |
2026.02 | 7.8 | - | - | |
2026.02 | 7.2 | - | - | |
2026.02 | 6.2 | - | - | |
2026.02 | 5.4 | - | - | |
2026.02 | 4 | - | - | |
2026.02 | 2.2 | - | - |