Long-Context LLM Inference
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| LLM Inference | Long-Context LLM Inference Decode | Latency (ms)0.13 | 8 | |
| LLM Inference | Long-Context LLM Inference (Prefill) | Prefill Latency (ms)0.62 | 6 |
Showing 2 of 2 rows
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| LLM Inference | Long-Context LLM Inference Decode | Latency (ms)0.13 | 8 | |
| LLM Inference | Long-Context LLM Inference (Prefill) | Prefill Latency (ms)0.62 | 6 |