LLM Prompt
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| LLM Prefill throughput | LLM Prompt Length 512 (pp512) | Throughput (tok/s)5,128 | 48 | |
| LLM Prefill throughput | LLM Prompt Length 256 (pp256) | Prefill Throughput (tok/s)5,063 | 48 | |
| LLM Prefill throughput | LLM Prompt Length 128 (pp128) | Throughput (tok/s)4,796 | 48 |
Showing 3 of 3 rows