Code Generation on MBPP 3-shot (Accuracy, TPS, Steps, Latency)
59.6Accuracy
KLASS
Updated 5d ago
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
2026.05 | 59.6 | 9.2 | 54 | 9.1 | |
2026.05 | 56.43 | 8.12 | - | 0.88 | |
2026.05 | 56.2 | 9.44 | - | 2.52 | |
2026.05 | 56 | 17.71 | - | 4.74 | |
2026.05 | 56 | 20.15 | - | 5.39 | |
2026.05 | 55 | 1 | 512 | 1 | |
2026.05 | 54.8 | 17.8 | 27 | 18.1 | |
2026.05 | 54.8 | 3.74 | - | 1 | |
2026.05 | 54.6 | 10 | 34 | 10.1 | |
2026.05 | 54.4 | 5.35 | - | 1.43 | |
2026.05 | 54.2 | 29.06 | - | 3.17 | |
2026.05 | 54 | 20.64 | - | 5.52 | |
2026.05 | 53.73 | 33.2 | - | 3.62 | |
2026.05 | 53.2 | 25.35 | - | 2.76 | |
2026.05 | 53 | 25.2 | 18 | 25 | |
2026.05 | 52.8 | 9.18 | - | 1 | |
2026.05 | 49.2 | 17.02 | - | 1.85 | |
2026.05 | 40.2 | 15.2 | 67 | 7.6 | |
2026.05 | 38.8 | 16.86 | - | 8.47 | |
2026.05 | 38.6 | 1.99 | - | 1 | |
2026.05 | 38.6 | 3.89 | - | 1.95 | |
2026.05 | 38 | 11.78 | - | 5.92 | |
2026.05 | 37.4 | 16.27 | - | 8.18 | |
2026.05 | 37 | 1 | 512 | 1 | |
2026.05 | 37 | 21.6 | 22 | 20.3 | |
2026.05 | 36.2 | 11.9 | 45 | 11.3 | |
2026.05 | 32 | 8.08 | - | 1.48 | |
2026.05 | 31.96 | 15.62 | - | 2.87 | |
2026.05 | 31.8 | 20.01 | - | 3.67 | |
2026.05 | 31.6 | 5.45 | - | 1 | |
2026.05 | 30.6 | 26.53 | - | 4.87 | |
2026.05 | 30.2 | 26.66 | - | 4.89 | |
2026.05 | 25.8 | 1.45 | - | 0.73 |