Decoding Latency on Llama-2-7B 32k sequence length v1 (inference)
0.062Decoding Latency (s)
TailorKV
Updated 1mo ago
Evaluation Results
| Method | Links | |
|---|---|---|
2025.05 | 0.062 | |
2025.05 | 0.077 | |
2025.05 | 0.087 | |
2025.05 | 0.111 | |
2025.05 | 0.838 | |
2025.05 | 1.776 |
TailorKV
| Method | Links | |
|---|---|---|
2025.05 | 0.062 | |
2025.05 | 0.077 | |
2025.05 | 0.087 | |
2025.05 | 0.111 | |
2025.05 | 0.838 | |
2025.05 | 1.776 |