Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Decode Throughput on Llama 3B 3.2
Loading...
117.3
Throughput (tok/s)
BaseRT
63.012
77.106
91.2
105.294
Jul 1, 2026
Throughput (tok/s)
Speedup vs. llama.cpp
Speedup vs. MLX
Updated 24d ago
Evaluation Results
Method
Method
Links
Throughput (tok/s)
Speedup vs. llama.cpp
Speedup vs. MLX
BaseRT
Quantization=Q4, Hardw...
2026.07
117.3
1.15
1.05
MLX
Quantization=Q4, Hardw...
2026.07
112.1
-
-
llama.cpp
Quantization=Q4, Hardw...
2026.07
102.4
-
-
BaseRT
Quantization=Q8, Hardw...
2026.07
70.9
1.09
1.08
MLX
Quantization=Q8, Hardw...
2026.07
65.5
-
-
llama.cpp
Quantization=Q8, Hardw...
2026.07
65.1
-
-
Feedback
Search any
task
Search any
task