Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Decode Throughput on Llama 3.2 1B
Loading...
295.4
Throughput (TOK/s)
BaseRT
153.752
190.526
227.3
264.074
Jul 1, 2026
Throughput (TOK/s)
Speedup vs. llama.cpp
Speedup vs. MLX
Updated 24d ago
Evaluation Results
Method
Method
Links
Throughput (TOK/s)
Speedup vs. llama.cpp
Speedup vs. MLX
BaseRT
Quantization=Q4, Hardw...
2026.07
295.4
1.28
1.15
MLX
Quantization=Q4, Hardw...
2026.07
257.8
-
-
llama.cpp
Quantization=Q4, Hardw...
2026.07
230.4
-
-
BaseRT
Quantization=Q8, Hardw...
2026.07
183.8
1.14
1.15
llama.cpp
Quantization=Q8, Hardw...
2026.07
160.7
-
-
MLX
Quantization=Q8, Hardw...
2026.07
159.2
-
-
Feedback
Search any
task
Search any
task