Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Stage-aware Prefill

Benchmarks

Task NameDataset NameSOTA ResultTrend
PrefillStage-aware Prefill
TTFT (ms)63.29
32
Text GenerationStage-aware Prefill 128K Prompt
TPOT (ms/token)38.64
4
Text GenerationStage-aware Prefill 64K Prompt
TPOT (ms/token)25.52
4
Text GenerationStage-aware Prefill 32K Prompt
Latency (ms/token)24.03
4
Text GenerationStage-aware Prefill 16K Prompt
TPOT (ms/token)23.97
4
Text GenerationStage-aware Prefill 8K Prompt
TPOT (ms/token)24.14
4
Text GenerationStage-aware Prefill 4K Prompt
TPOT (ms/token)24.15
4
Text GenerationStage-aware Prefill 2K Prompt
TPOT (ms/token)24.11
4
Text GenerationStage-aware Prefill 1K Prompt
TPOT (ms/token)24.05
4
Showing 9 of 9 rows