Long-Context
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| LLM Decoding | Long Context 32K | Decode Throughput (tok/s)1,218.1 | 48 | |
| LLM Decoding | Long Context 64K | Decoding Throughput (tok/s)1,042.6 | 42 | |
| LLM Decoding | Long Context 96K | Decode Throughput (tok/s)930.5 | 38 | |
| LLM Decoding | Long Context 128K | Throughput (tok/s)876.9 | 33 | |
| Many-shot in-context learning | Long-context benchmarks | ICL Performance (8k Context)74.2 | 21 | |
| Context Management | Long-context (test) | mTokens1 | 19 | |
| Long Context | Long Context benchmark | Accuracy67.59 | 14 | |
| Tabular Learning | Long-context 15 datasets v2 (test) | Avg. Normalized RMSE0.523 | 9 | |
| Average across tasks | Long-context benchmarks | Performance (8k Context)45.9 | 8 | |
| End-to-end LLM Inference Serving | Long-context 1024-token input, 32-token output | TPOT Speedup vs DeepGEMM1.48 | 3 | |
| Long-Context Training | Long-Context (train) | Metric- | 0 |
Showing 11 of 11 rows