NIAH

Benchmarks

Task Name	Dataset Name	SOTA Result
Multi-needle retrieval	NIAH (M)	Accuracy (NIAH M)90.2	35
Long-context retrieval	NIAH multivalue	Speedup4.1	20
Long Context Retrieval	NIAH-Multi	Accuracy100	13
Long-context retrieval	NIAH (avg)	Score (4k Context)100	7
Long Context	NIAH	Accuracy99.8	6
Long-context retrieval	NIAH 32k	NIAH Score99	6
Long-context retrieval	NIAH 16k	NIAH Score98.6	6
Needle-in-a-haystack	NIAH Needle-in-a-haystack	NIAH Success Rate (32K Context)100	6
Needle-in-a-haystack	NIAH 1	Success Rate (1k Context)79.69	5
Needle-in-a-haystack	NIAH-2 (test)	NIAH-2 Success Rate (1k)79.61	5
Long-context recall	NIAH Single-3	Recall @ 32K Context100	4
Long-context recall	NIAH Single 2	Recall @ 32K Context1	4
Long-context recall	NIAH Single-1	Recall @ 32K100	4

Showing 13 of 13 rows