Language Model Inference Efficiency on Meta-Llama-3-8B
1,398Throughput (tokens/s)
AWQ
Updated 1mo ago
Evaluation Results
| Method | Links | ||
|---|---|---|---|
2026.02 | 1,398 | 0.715 | |
2026.02 | 1,022.5 | 0.978 | |
2026.02 | 975.05 | 1.025 | |
2026.02 | 857.6 | 1.166 |
AWQ
| Method | Links | ||
|---|---|---|---|
2026.02 | 1,398 | 0.715 | |
2026.02 | 1,022.5 | 0.978 | |
2026.02 | 975.05 | 1.025 | |
2026.02 | 857.6 | 1.166 |