vLLM serving benchmark
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Large Language Model Serving | vLLM serving benchmark 128 prompts, 32 pre-fill tokens, 256 generation tokens | TTFT (ms)76 | 9 |
Showing 1 of 1 rows
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Large Language Model Serving | vLLM serving benchmark 128 prompts, 32 pre-fill tokens, 256 generation tokens | TTFT (ms)76 | 9 |