Needle-In-A-Haystack Retrieval

Benchmarks

Dataset Name	SOTA Method	Metric
RULER	Full attention	S-NIAH-1 (Pass-Key Retrieval)100	94	2mo ago
Needle-in-a-Haystack 32K context (test)	Quest	Accuracy76	30	4mo ago
Needle-in-a-Haystack 8K context (test)	Quest	Accuracy100	30	4mo ago
NIAH-Single-3 256K context	FlashMorph	Accuracy94	23	25d ago
NIAH-Single-3 128K context	Qwen3+YaRN	Accuracy98.8	23	25d ago
NIAH-Single-3 64K context	Qwen3	Accuracy100	23	25d ago
NIAH-Single-3 32K context	Qwen3	Accuracy100	23	25d ago
NIAH-Single-2 256K context	HALO	Accuracy95.2	23	25d ago
NIAH-Single 128K context 2	Qwen3+YaRN	Accuracy98.6	23	25d ago
NIAH-Single-2 32K context	Qwen3	Accuracy100	23	25d ago
NIAH-Single-1 256K context		Accuracy100	23	25d ago
NIAH-Single-1 128K context	HALO	Accuracy100	23	25d ago
NIAH-Single-1 64K context	Qwen3	Accuracy100	23	25d ago
NIAH-Single-1 32K context	Qwen3	Accuracy (NIAH-Single-1 32K)100	23	25d ago
NIAH	Transformer	NIAH Score100	14	1mo ago
NIAH Single 64K context 2	FlashMorph	Accuracy100	13	25d ago
NIAH 64K 60 items, 3 needle positions (test)		F1 Score28.2	8	2mo ago
RULER (test)	DroPE transformer	Multi-Query Success Rate2,800	8	4mo ago
NIAH L=2048	real_screen	Accuracy100	6	2mo ago
RULER NIAH		Retrieval Success (1K)100	4	2mo ago
RULER S-NIAH-2 OOD	ASEntmax	Success Rate (4K Context)83.2	4	4mo ago
RULER S-NIAH-2 (ID)	Softmax	Retrieval Success Rate (1K)100	4	4mo ago
RULER S-NIAH-1 OOD	ASEntmax	Success Rate (4K Context)100	4	4mo ago
RULER S-NIAH-1 ID	Softmax	Retrieval Success Rate (1K Context)100	4	4mo ago
BABILong 32K context length	MemDLM (Train & Inference)	Accuracy9	3	4mo ago

Showing 25 of 29 rows