Decoding Latency on Llama-2-7B 96k sequence length v1 (inference)
0.115Decoding Latency (s)
PQCache
Updated 4d ago
Evaluation Results
| Method | Links | |
|---|---|---|
2025.05 | 0.115 | |
2025.05 | 0.132 | |
2025.05 | 0.176 | |
2025.05 | 3.253 |
PQCache
| Method | Links | |
|---|---|---|
2025.05 | 0.115 | |
2025.05 | 0.132 | |
2025.05 | 0.176 | |
2025.05 | 3.253 |