Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Decode Throughput on BenchRandom
Loading...
269.1
Decode Throughput (tok/s)
NVE
6.604
74.752
142.9
211.048
Apr 22, 2026
Decode Throughput (tok/s)
Speedup
Updated 1mo ago
Evaluation Results
Method
Method
Links
Decode Throughput (tok/s)
Speedup
NVE
Model=Llama-3.2-1B, Qu...
2026.04
269.1
-
llama.cpp
Model=Llama-3.2-1B, Qu...
2026.04
150.8
-
NVE
Model=Llama-3.2-1B, Qu...
2026.04
116.7
-
NVE
Model=Llama-3.2-3B, Qu...
2026.04
108.8
-
llama.cpp
Model=Llama-3.2-3B, Qu...
2026.04
70.9
-
NVE
Model=Llama-3.1-8B, Qu...
2026.04
47.7
-
NVE
Model=Llama-3.2-3B, Qu...
2026.04
43
-
llama.cpp
Model=Llama-3.1-8B, Qu...
2026.04
30.8
-
NVE
Model=Llama-3.1-8B, Qu...
2026.04
16.7
-
Feedback
Search any
task
Search any
task
