Context sequences
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Time to First Token (TTFT) measurement | Context sequences 32K-256K vLLM 0.18.0 / 0.20.0 (online-serving) | TTFT (Dense, s)2.944 | 7 |
Showing 1 of 1 rows
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Time to First Token (TTFT) measurement | Context sequences 32K-256K vLLM 0.18.0 / 0.20.0 (online-serving) | TTFT (Dense, s)2.944 | 7 |