Text Generation Throughput on R1-Llama-8B 32K generation
2.51Memory Footprint (%)
ThinKV
Updated 23d ago
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
2025.10 | 2.51 | 711 | 8,412.2 | 938 | 10,578.5 | |
2025.10 | 5.48 | 268 | 1,450.5 | 350 | 2,425.8 | |
2025.10 | 5.48 | 268 | 2,320.9 | 350 | 4,311.3 | |
2025.10 | 5.48 | 256 | 1,769.3 | 256 | 2,489.8 | |
2025.10 | 5.48 | 256 | 3,539.3 | 256 | 5,318.7 | |
2025.10 | 5.78 | 256 | 5,298.4 | 256 | 8,079.9 | |
2025.10 | 100 | 13 | 297.5 | 19 | 453.9 |