Multi-task Language Understanding on MMLU Pro (Throughput and Latency)
10.12Throughput
d2Cache
Updated 1mo ago
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
2025.09 | 10.12 | 25.77 | 46.8 | |
2025.09 | 9.74 | 27.69 | 45.9 | |
2025.09 | 9.71 | 26.73 | 44.4 | |
2025.09 | 9.59 | 27.6 | 33.1 | |
2025.09 | 9.58 | 27.6 | 39.1 | |
2025.09 | 9.42 | 27.31 | 45.9 | |
2025.09 | 8.96 | 28.83 | 40 | |
2025.09 | 8.91 | 29 | 37.1 | |
2025.09 | 7.82 | 34.09 | 46.5 | |
2025.09 | 7.45 | 34.81 | 44.6 | |
2025.09 | 6.86 | 37.96 | 37.4 | |
2025.09 | 6.79 | 38.29 | 38.1 | |
2025.09 | 2.15 | 126.31 | 47.9 | |
2025.09 | 2.13 | 127.08 | 46.1 | |
2025.09 | 1.76 | 152.62 | 37.5 | |
2025.09 | 1.53 | 143.45 | 38.1 |