Inference Throughput on Llama Instruct 3.1-8B (internal harness)
6,991Throughput (TPS)
Pre-compressed only
Updated 1mo ago
Evaluation Results
| Method | Links | |
|---|---|---|
2026.03 | 6,991 | |
2026.03 | 6,872 | |
2026.03 | 4,606 | |
2026.03 | 3,316 | |
2026.03 | 2,194 | |
2026.03 | 2,194 | |
2026.03 | 1,812 | |
2026.03 | 1,023 |