Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Large Language Model Inference on Llama-3-8B
Loading...
130.7
Throughput (Tok/s)
GPTQ
57.5672
76.5536
95.54
114.5264
Jun 9, 2026
Throughput (Tok/s)
Peak GMEM (GB)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Throughput (Tok/s)
Peak GMEM (GB)
GPTQ
Hardware=Single NVIDIA...
2026.06
130.7
4.14
UniSVQ
Hardware=Single NVIDIA...
2026.06
101.65
3.87
QUIP#
Hardware=Single NVIDIA...
2026.06
79.6
4.13
AQLM
Hardware=Single NVIDIA...
2026.06
70.97
4.44
FP16
Hardware=Single NVIDIA...
2026.06
60.38
15.72
Feedback
Search any
task
Search any
task
