Decoding Latency on Llama-3.1-8B 64k sequence length v1
0.05Decoding Latency (s)
Full Cache
Updated 1mo ago
Evaluation Results
| Method | Links | |
|---|---|---|
2025.05 | 0.05 | |
2025.05 | 0.054 | |
2025.05 | 0.074 | |
2025.05 | 0.108 | |
2025.05 | 0.435 |
Full Cache
| Method | Links | |
|---|---|---|
2025.05 | 0.05 | |
2025.05 | 0.054 | |
2025.05 | 0.074 | |
2025.05 | 0.108 | |
2025.05 | 0.435 |