Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Latency profiling on Llama-3.1-8B
Loading...
7.3
TPOT
UniQL
6.592
11.371
16.15
20.929
Dec 3, 2025
TPOT
TTLT
Updated 1mo ago
Evaluation Results
Method
Method
Links
TPOT
TTLT
UniQL
W-bit=4, Prun. p%=35%
2025.12
7.3
8,105.4
UniQL
W-bit=4, Prun. p%=0%
2025.12
9
9,944.6
TAO-HQQ
W-bit=4*, Prun. p%=0%
2025.12
10.2
11,639.5
TRT-AWQ
W-bit=4*, Prun. p%=0%
2025.12
11.2
11,665.7
FP16
W-bit=16, Prun. p%=0%
2025.12
25
26,653.8
Feedback
Search any
task
Search any
task
