Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Inference Serving Performance on 1024 Input / 32 Output Context
Loading...
1.48
TPOT Speedup vs DeepGEMM
RaMP
1.2304
1.2952
1.36
1.4248
Apr 28, 2026
TPOT Speedup vs DeepGEMM
TPOT Speedup vs Triton
TPOT Speedup vs FlashInfer
TTFT Speedup vs DeepGEMM
TTFT Speedup vs Triton
TTFT Speedup vs FlashInfer
Updated 1mo ago
Evaluation Results
Method
Method
Links
TPOT Speedup vs DeepGEMM
TPOT Speedup vs Triton
TPOT Speedup vs FlashInfer
TTFT Speedup vs DeepGEMM
TTFT Speedup vs Triton
TTFT Speedup vs FlashInfer
RaMP
Request rate (r)=4, Mo...
2026.04
1.48
1.34
1.16
1.12
1.19
1.06
RaMP
Request rate (r)=2, Mo...
2026.04
1.43
1.34
1.15
1.04
1.18
1.08
RaMP
Request rate (r)=8, Mo...
2026.04
1.24
1.29
1.11
0.95
1.13
1.02
Feedback
Search any
task
Search any
task
