Decoding Latency on Llama-2-7B 16k sequence length v1 (inference)
0.041Decoding Latency (s)
TailorKV
Updated 4d ago
Evaluation Results
| Method | Links | |
|---|---|---|
2025.05 | 0.041 | |
2025.05 | 0.045 | |
2025.05 | 0.067 | |
2025.05 | 0.108 | |
2025.05 | 0.433 | |
2025.05 | 0.893 |
TailorKV
| Method | Links | |
|---|---|---|
2025.05 | 0.041 | |
2025.05 | 0.045 | |
2025.05 | 0.067 | |
2025.05 | 0.108 | |
2025.05 | 0.433 | |
2025.05 | 0.893 |