Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Text Generation on Decoding Throughput
Loading...
320
Decoding Throughput (tokens/s)
AWQ
13.2
92.85
172.5
252.15
Nov 13, 2025
Decoding Throughput (tokens/s)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Decoding Throughput (tokens/s)
AWQ
Model=Qwen3-1.7B, Batc...
2025.11
320
PAROQ
Model=Qwen3-1.7B, Batc...
2025.11
278
QTIP
Model=Qwen3-1.7B, Batc...
2025.11
209
AWQ
Model=Qwen3-4B, Batch...
2025.11
176
FP16
Model=Qwen3-1.7B, Batc...
2025.11
170
PAROQ
Model=Qwen3-4B, Batch...
2025.11
160
AWQ
Model=LLaMA-3-8B, Batc...
2025.11
120
QTIP
Model=Qwen3-4B, Batch...
2025.11
117
PAROQ
Model=LLaMA-3-8B, Batc...
2025.11
112
QTIP
Model=LLaMA-3-8B, Batc...
2025.11
95
FP16
Model=Qwen3-4B, Batch...
2025.11
78
AWQ
Model=Qwen3-14B, Batch...
2025.11
70
PAROQ
Model=Qwen3-14B, Batch...
2025.11
65
QTIP
Model=Qwen3-14B, Batch...
2025.11
55
FP16
Model=LLaMA-3-8B, Batc...
2025.11
45
FP16
Model=Qwen3-14B, Batch...
2025.11
25
Feedback
Search any
task
Search any
task
