Code Generation on HumanEval (Throughput, Speedup, and Avg. τ)
1.94Average Tau (τ)
MEDUSA
Updated 4d ago
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
2025.12 | 1.94 | - | 1.34 | |
2025.12 | 1.97 | - | 1.32 | |
2025.12 | 2.53 | - | 2.3 | |
2025.12 | 2.57 | - | 2.14 | |
2025.12 | 2.58 | - | 2.39 | |
2025.12 | 2.61 | - | 1.91 | |
2025.12 | 2.63 | - | 2.01 | |
2026.02 | 3.01 | - | 1.74 | |
2026.02 | 3.05 | - | 1.84 | |
2026.02 | 3.05 | - | 1.75 | |
2026.02 | 3.17 | - | 1.89 | |
2025.12 | 3.21 | - | 2.41 | |
2025.12 | 3.23 | - | 2.18 | |
2025.12 | 3.25 | - | 2.41 | |
2025.12 | 3.26 | - | 2.47 | |
2025.12 | 3.34 | - | 2.65 | |
2026.02 | 3.39 | - | 1.94 | |
2026.02 | 3.47 | - | 2.12 | |
2026.02 | 3.54 | - | 2.05 | |
2026.02 | 3.65 | - | 2.17 | |
2025.12 | 3.72 | - | 2.62 | |
2025.12 | 3.87 | - | 2.65 | |
2025.12 | 3.9 | - | 2.52 | |
2025.12 | 4.21 | - | 2.79 | |
2025.12 | 4.51 | - | 2.59 | |
2026.02 | 4.57 | 22.53 | - | |
2025.12 | 4.66 | - | 3.18 | |
2025.12 | 4.68 | - | 2.97 | |
2025.12 | 5 | - | 3.3 | |
2025.12 | 5 | - | 3.5 | |
2026.02 | 5.13 | 21.17 | - | |
2026.02 | 5.39 | 21.2 | - | |
2025.12 | 5.48 | - | 4.32 | |
2026.02 | 5.52 | - | 4.32 | |
2025.12 | 5.72 | - | 3.59 | |
2026.02 | 6.04 | - | 4.74 | |
2026.02 | 6.5 | 955 | 4.2 | |
2026.02 | 6.5 | - | 5.14 | |
2025.12 | 6.58 | - | 5.54 | |
2026.02 | 6.63 | 1,247 | 4 | |
2026.02 | 6.64 | - | 5.21 | |
2026.02 | 7.46 | 24.2 | - | |
2026.02 | 8.09 | 802 | 3.5 | |
2026.02 | 8.36 | 49.05 | 2.71 | |
2026.02 | 10.9 | 51.07 | 2.82 | |
2026.02 | - | 312 | - | |
2026.02 | - | 229 | - | |
2026.02 | - | 229 | - | |
2026.02 | - | 18.1 | 1 |