Inference Efficiency on 90k Context Length Llama-3.1-8B
8.9Throughput (queries/s)
Finetuning
Updated 3d ago
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
2025.03 | 8.9 | - | - | - | |
2025.03 | 7.7 | - | - | - | |
2025.03 | 6.8 | - | - | - | |
2025.03 | 0.4 | - | - | - | |
2025.03 | - | - | 1 | 1 | |
2025.03 | - | - | 6.5 | 0.06 | |
2025.03 | - | - | - | 0.046 | |
2025.03 | - | - | 4 | 0.053 |