| Dataset Name | SOTA Method | Metric | Trend | ||
|---|---|---|---|---|---|
| Llama-3.1-8B-Instruct 128K input length | TriangleMix + MInference | TTFT (s)14.5 | 9 | 1mo ago | |
| Llama 8B Instruct 112K input length 3.1 | TriangleMix + MInference | Time-to-First-Token (s)12.7 | 9 | 1mo ago | |
| Llama-3.1-8B-Instruct 96K input length | TriangleMix + MInference | TTFT (s)10.9 | 9 | 1mo ago | |
| Llama-3.1-8B-Instruct 80K input length | TriangleMix + FlexPrefill | TTFT (s)9.2 | 9 | 1mo ago | |
| Llama 8B Instruct 64K input length 3.1 | TriangleMix + FlexPrefill | TTFT (s)7.2 | 9 | 1mo ago | |
| Llama 8B Instruct 48K input length 3.1 | TriangleMix + FlexPrefill | TTFT (s)5.2 | 9 | 1mo ago | |
| Llama 8B Instruct 32K input length 3.1 | TriangleMix + FlexPrefill | TTFT (s)3.4 | 9 | 1mo ago | |
| GSM8K | Behavior-Equivalent Token | TTFT (ms)19.99 | 8 | 3mo ago | |
| RoleLLM | Behavior-Equivalent Token | TTFT (ms)19.17 | 8 | 3mo ago |
