Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
LLM Serving on ToolBench
Loading...
1.11
Effective Throughput (req/s)
AugServe
0.0596
0.3323
0.605
0.8777
Dec 3, 2025
Effective Throughput (req/s)
Updated 4d ago
Evaluation Results
Method
Method
Links
Effective Throughput (req/s)
AugServe
Model=OPT-13B, GPU=H80...
2025.12
1.11
vLLM
Model=OPT-13B, GPU=H80...
2025.12
1.09
AugServe
Model=OPT-13B, GPU=H80...
2025.12
0.71
AugServe
Model=OPT-13B, GPU=H80...
2025.12
0.56
InferCept
Model=OPT-13B, GPU=H80...
2025.12
0.31
vLLM
Model=OPT-13B, GPU=H80...
2025.12
0.25
vLLM
Model=OPT-13B, GPU=H80...
2025.12
0.19
InferCept
Model=OPT-13B, GPU=H80...
2025.12
0.16
InferCept
Model=OPT-13B, GPU=H80...
2025.12
0.1
Feedback
Search any
task
Search any
task
