Stage-aware Prefill
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Prefill | Stage-aware Prefill | TTFT (ms)63.29 | 32 | |
| Text Generation | Stage-aware Prefill 128K Prompt | TPOT (ms/token)38.64 | 4 | |
| Text Generation | Stage-aware Prefill 64K Prompt | TPOT (ms/token)25.52 | 4 | |
| Text Generation | Stage-aware Prefill 32K Prompt | Latency (ms/token)24.03 | 4 | |
| Text Generation | Stage-aware Prefill 16K Prompt | TPOT (ms/token)23.97 | 4 | |
| Text Generation | Stage-aware Prefill 8K Prompt | TPOT (ms/token)24.14 | 4 | |
| Text Generation | Stage-aware Prefill 4K Prompt | TPOT (ms/token)24.15 | 4 | |
| Text Generation | Stage-aware Prefill 2K Prompt | TPOT (ms/token)24.11 | 4 | |
| Text Generation | Stage-aware Prefill 1K Prompt | TPOT (ms/token)24.05 | 4 |
Showing 9 of 9 rows