End-to-end decode throughput on Context Length 32K
183Decode Throughput (tok/s)
Llama-3.1-8B
Updated 14d ago
Evaluation Results
| Method | Links | |
|---|---|---|
2026.05 | 183 | |
2026.05 | 172.5 | |
2026.05 | 153.9 | |
2026.05 | 110 | |
2026.05 | 104.9 | |
2026.05 | 93.3 |
Llama-3.1-8B
| Method | Links | |
|---|---|---|
2026.05 | 183 | |
2026.05 | 172.5 | |
2026.05 | 153.9 | |
2026.05 | 110 | |
2026.05 | 104.9 | |
2026.05 | 93.3 |