End-to-end decode throughput on 128K Context Length
108Throughput (tok/s)
Llama-3.1-8B
Updated 14d ago
Evaluation Results
| Method | Links | |
|---|---|---|
2026.05 | 108 | |
2026.05 | 102.4 | |
2026.05 | 94.8 | |
2026.05 | 62.9 | |
2026.05 | 59.9 | |
2026.05 | 55.7 |
Llama-3.1-8B
| Method | Links | |
|---|---|---|
2026.05 | 108 | |
2026.05 | 102.4 | |
2026.05 | 94.8 | |
2026.05 | 62.9 | |
2026.05 | 59.9 | |
2026.05 | 55.7 |