Time-to-first-token (TTFT) on Llama-3.1-8B-Instruct 128K input length
14.5TTFT (s)
TriangleMix + MInference
Updated 1mo ago
Evaluation Results
| Method | Links | |
|---|---|---|
2025.07 | 14.5 | |
2025.07 | 16.1 | |
2025.07 | 17.8 | |
2025.07 | 17.8 | |
2025.07 | 18 | |
2025.07 | 23.2 | |
2025.07 | 23.4 | |
2025.07 | 23.7 | |
2025.07 | 34.5 |