Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Inference Scheduling on LMSYS-Chat-1M
Loading...
2.41
Average Per-token Latency (s/token)
TIE
2.1432
3.9441
5.745
7.5459
Apr 1, 2026
Average Per-token Latency (s/token)
P90 Per-token Latency (s/token)
Average TTFT (s)
P90 TTFT (s)
Updated 17d ago
Evaluation Results
Method
Method
Links
Average Per-token Latency (s/token)
P90 Per-token Latency (s/token)
Average TTFT (s)
P90 TTFT (s)
TIE
Testing Model Size=70B...
2026.04
2.41
4.05
204.03
475.1
LTR
Testing Model Size=70B...
2026.04
4.34
7.03
252.2
507.35
SSJF
Testing Model Size=70B...
2026.04
5.5
8.24
273.3
551.15
FCFS
Testing Model Size=70B...
2026.04
9.08
16.13
319.51
618.21
Feedback
Search any
task
Search any
task
