Context Length
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Inference Efficiency | 90k Context Length Llama-3.1-8B | Throughput (queries/s)8.9 | 4 | |
| Inference Efficiency | 30k Context Length (Llama-3.1-8B) | Inference Throughput (QPS)15.8 | 4 | |
| Inference Efficiency | 30k Context Length Llama-2-7B | Inference Throughput (QPS)6.6 | 4 |
Showing 3 of 3 rows