End-to-end decode throughput on 8K Context Length
268.5Throughput (tok/s)
Llama-3.1-8B
Updated 14d ago
Evaluation Results
| Method | Links | |
|---|---|---|
2026.05 | 268.5 | |
2026.05 | 261.2 | |
2026.05 | 239.4 | |
2026.05 | 173.4 | |
2026.05 | 163.7 | |
2026.05 | 151.6 |
Llama-3.1-8B
| Method | Links | |
|---|---|---|
2026.05 | 268.5 | |
2026.05 | 261.2 | |
2026.05 | 239.4 | |
2026.05 | 173.4 | |
2026.05 | 163.7 | |
2026.05 | 151.6 |