Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Serving Efficiency on LMSYS trace
Loading...
246
GPUs Used
Token-budget
241.52
271.76
302
332.24
Apr 9, 2026
GPUs Used
Cost Savings
P99 TTFT (s)
Updated 9d ago
Evaluation Results
Method
Method
Links
GPUs Used
Cost Savings
P99 TTFT (s)
Token-budget
Request Rate=1,000 req...
2026.04
246
31.3
1.48
Homogeneous
Request Rate=1,000 req...
2026.04
358
-
1.45
Feedback
Search any
task
Search any
task
