LLM Inference on LLaMA-7B v1 (serving)
12.16Decode Latency (ms/token)
FlashSVD v1.5
Updated 22d ago
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
2026.05 | 12.16 | 0.43 | 2.55 | |
2026.05 | 12.16 | 0.43 | 2.2 | |
2026.05 | 12.24 | 1.72 | 2.5 | |
2026.05 | 12.24 | 1.72 | 2.13 | |
2026.05 | 12.85 | 1.93 | 2.38 | |
2026.05 | 12.85 | 1.93 | 2.07 | |
2026.05 | 14.09 | 2.39 | 2.18 | |
2026.05 | 14.09 | 2.39 | 1.86 | |
| 26.19 | 3.51 | - | ||
| 26.21 | 3.94 | - | ||
| 26.52 | 3.68 | - | ||
| 26.9 | 0.91 | - | ||
| 30.6 | 4.29 | - | ||
| 30.63 | 4.1 | - | ||
| 30.65 | 4.85 | - | ||
| 30.84 | 1.03 | - |