Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Latency and Throughput on MultiNews, Qasper, RepoBench-P, and RULER Averaged 128K (test)
Loading...
15.3
Memory Footprint (GB)
TTKV
12.232
32.941
53.65
74.359
Mar 27, 2026
Memory Footprint (GB)
Average Latency (ms)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Memory Footprint (GB)
Average Latency (ms)
TTKV
Model=Llama-3.1-70B
2026.03
15.3
595
TTKV
Model=Qwen2.5-32B
2026.03
15.5
555
TTKV
Model=DeepSeek-R1-14B
2026.03
15.8
465
FP16
Model=Llama-3.1-70B
2026.03
92
2,450
FP16
Model=Qwen2.5-32B
2026.03
92
2,350
FP16
Model=DeepSeek-R1-14B
2026.03
92
2,550
Feedback
Search any
task
Search any
task
