Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
LLM Serving on Merge dataset
Loading...
0.6
Effective Throughput (req/s)
AugServe
0.0904
0.2227
0.355
0.4873
Dec 3, 2025
Effective Throughput (req/s)
Updated 1mo ago
Evaluation Results
Method
Method
Links
Effective Throughput (req/s)
AugServe
Model=OPT-13B, GPU=H80...
2025.12
0.6
AugServe
Model=OPT-13B, GPU=H80...
2025.12
0.57
AugServe
Model=OPT-13B, GPU=H80...
2025.12
0.51
InferCept
Model=OPT-13B, GPU=H80...
2025.12
0.27
vLLM
Model=OPT-13B, GPU=H80...
2025.12
0.22
InferCept
Model=OPT-13B, GPU=H80...
2025.12
0.21
InferCept
Model=OPT-13B, GPU=H80...
2025.12
0.16
vLLM
Model=OPT-13B, GPU=H80...
2025.12
0.15
vLLM
Model=OPT-13B, GPU=H80...
2025.12
0.11
Feedback
Search any
task
Search any
task
