S-NIAH
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Long-context retrieval | S-NIAH | Latency (s)10.5 | 27 | |
| Semantic Needle-In-A-Haystack | S-NIAH | Accuracy52.4 | 27 | |
| Number in haystack retrieval | S-NIAH-2 number | Accuracy (1K Context)100 | 14 | |
| Single-Needle-in-a-Haystack | S-NIAH (test) | Exact Match Accuracy100 | 12 | |
| Long-context Retrieval | S-NIAH | Exact Match Accuracy99.6 | 12 | |
| Synthetic Retrieval | S-NIAH | Score98.9 | 4 |
Showing 6 of 6 rows