64K/64K serving scenario
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Max token throughput | 64K/64K serving scenario 8xH100 node 1.0 | Max Throughput (K tok/s)9.3 | 5 |
Showing 1 of 1 rows
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Max token throughput | 64K/64K serving scenario 8xH100 node 1.0 | Max Throughput (K tok/s)9.3 | 5 |