LLM Inference on LLaMA-2 70B sequence length 2048
384Max Batch Size
CXL-SpecKV + Comp
Updated 1mo ago
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
2025.12 | 384 | - | 24 | |
2025.12 | 192 | - | 12 | |
2025.12 | 128 | - | 8 | |
2025.12 | 48 | - | 3 | |
2025.12 | 16 | - | 1 |
CXL-SpecKV + Comp
| Method | Links | |||
|---|---|---|---|---|
2025.12 | 384 | - | 24 | |
2025.12 | 192 | - | 12 | |
2025.12 | 128 | - | 8 | |
2025.12 | 48 | - | 3 | |
2025.12 | 16 | - | 1 |