Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Serving on Synthetic Workloads (Maximum Serving Capacity)
Loading...
12.8
Throughput (req/s)
DuetServe
6.664
8.257
9.85
11.443
Nov 6, 2025
Throughput (req/s)
Mean TBT (ms)
Throughput Gain (×)
Updated 1d ago
Evaluation Results
Method
Method
Links
Throughput (req/s)
Mean TBT (ms)
Throughput Gain (×)
DuetServe
ISL=4096, OSL=64, ISL/...
2025.11
12.8
105
1.28
vLLM
ISL=4096, OSL=64, ISL/...
2025.11
10
170
-
DuetServe
ISL=4096, OSL=1024, IS...
2025.11
9.8
50
1.11
vLLM
ISL=4096, OSL=1024, IS...
2025.11
8.8
55
-
DuetServe
ISL=4096, OSL=2048, IS...
2025.11
7.2
44
1.04
vLLM
ISL=4096, OSL=2048, IS...
2025.11
6.9
45
-
Feedback
Search any
task
Search any
task
