| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| NVFP4-W4A16 | FlashRT | Time to First Token (ms)182 | 32 | 1mo ago | |
| LLM Prefix-based Inference | FlashRT | TTFT (ms)51 | 12 | 1mo ago | |
| Llama-3.1-8B-Instruct 128K input length | TriangleMix + MInference | TTFT (s)14.5 | 9 | 3mo ago | |
| Llama 8B Instruct 112K input length 3.1 | TriangleMix + MInference | Time-to-First-Token (s)12.7 | 9 | 3mo ago | |
| Llama-3.1-8B-Instruct 96K input length | TriangleMix + MInference | TTFT (s)10.9 | 9 | 3mo ago | |
| Llama-3.1-8B-Instruct 80K input length | TriangleMix + FlexPrefill | TTFT (s)9.2 | 9 | 3mo ago | |
| Llama 8B Instruct 64K input length 3.1 | TriangleMix + FlexPrefill | TTFT (s)7.2 | 9 | 3mo ago | |
| Llama 8B Instruct 48K input length 3.1 | TriangleMix + FlexPrefill | TTFT (s)5.2 | 9 | 3mo ago | |
| Llama 8B Instruct 32K input length 3.1 | TriangleMix + FlexPrefill | TTFT (s)3.4 | 9 | 3mo ago | |
| GSM8K | Behavior-Equivalent Token | TTFT (ms)19.99 | 8 | 4mo ago | |
| RoleLLM | Behavior-Equivalent Token | TTFT (ms)19.17 | 8 | 4mo ago | |
| Generalization Models Tulu3, Qwen3, Lego-Link0 | LazyAttention | TTFT (Standard, ms)1,195.6 | 3 | 1mo ago |
