NIAH

Benchmarks

Task Name	Dataset Name	SOTA Result
Information Retrieval	NIAH (test)	Average Score99.6	59
Multi-needle retrieval	NIAH (M)	Accuracy (NIAH M)90.2	35
Long-context Retrieval	NIAH 128k	Single Score24.4	20
Long-context Retrieval	NIAH 64k	Single Score49.3	20
Long-context retrieval	NIAH multivalue	Speedup4.1	20
Long Context Retrieval	NIAH-Multi	Accuracy100	13
Synthetic Retrieval	NIAH	Score @ 4096 Context100	9
Information Retrieval	NIAH single v1.0 (test)	Accuracy100	8
Long-context retrieval	NIAH (avg)	Score (4k Context)100	7
Long Context	NIAH	Accuracy99.8	6
Long-context retrieval	NIAH 32k	NIAH Score99	6
Long-context retrieval	NIAH 16k	NIAH Score98.6	6
Needle-in-a-haystack	NIAH Needle-in-a-haystack	NIAH Success Rate (32K Context)100	6
Long-context Retrieval	NIAH	Accuracy100	5
Needle-in-a-haystack	NIAH 1	Success Rate (1k Context)79.69	5
Needle-in-a-haystack	NIAH-2 (test)	NIAH-2 Success Rate (1k)79.61	5
Information Retrieval	NIAH multikey v1.0 (test)	Accuracy98.7	4
Long Context & Context Learning	NIAH@1M RULER	pass@199	4
Long-context recall	NIAH Single-3	Recall @ 32K Context100	4
Long-context recall	NIAH Single 2	Recall @ 32K Context1	4
Long-context recall	NIAH Single-1	Recall @ 32K100	4
Needle-In-A-Haystack Retrieval	NIAH Single 2	Success Rate (4K Context)94	2

Showing 22 of 22 rows