NOSA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| LLM Decoding Throughput | NOSA-8B 32K context length | Throughput (tok/s)2,312.1 | 24 | |
| LLM Decoding Throughput | NOSA-8B 64K context length | Throughput (tok/s)1,996.3 | 21 | |
| Prefill throughput | NOSA-8B | Prefill Throughput (tok/s)20,438.9 | 20 | |
| LLM Decoding Throughput | NOSA-8B 96K context length | Throughput (tok/s)1,694.1 | 19 | |
| LLM Decoding Throughput | NOSA-8B 128K context length | Throughput (tok/s)1,463.3 | 16 |
Showing 5 of 5 rows