SOTA Long-Context Retrieval on RULER 16K and PapersWithCode

98.67Score

HiLS-Attn HoPE-Q-Cal

Updated 19d ago

Evaluation Results

Method	Links
HiLS-Attn HoPE-Q-Cal 2026.07		98.67
Ministral-3-8B-Instruct-2512-BF16 2026.05		93.4
Llama-3.1-8B-Instruct 2026.05		93.3
Qwen3-8B 2026.05		92
GPT-5 nano 2026.05		91.7
Qwen3-4B 2026.05		91
Full Attention 2026.04		89.12
Hybrid-KSA 2026.04		88.86
gemma-3-12b-it 2026.05		87.5
KSA 2026.04		84.78
Llama-3.2-3B-Instruct 2026.05		83.1
Hybrid-SWA 2026.04		82.87
Hybrid-SCA 2026.04		80.67
Hybrid-Linear 2026.04		78.06
gpt-oss-20b 2026.05		77.2
Hybrid-KSA 2026.04		74.07
Hybrid-GDN 2026.04		74.04
HiLS-Attn NoPE-Q-Cal 2026.07		73.33
Full 2026.04		73.24
Hybrid-SCA 2026.04		72.55
Hybrid-SWA 2026.04		69.54
gemma-3-4b-it 2026.05		68.5
KSA 2026.04		66.74
FastwebMIIA-7B 2026.05		58.7
EngGPT2-16B-A3B 2026.05		57.9
Moonlight-16B-A3B-Instruct 2026.05		55.7
HiLS-Attn RoPE-Q-Cal 2026.07		50.67
LLaMAntino-3-ANITA-8B-Inst-DPO-ITA 2026.05		48.2
Velvet-14B 2026.05		34.4
Olmo3 512swa 2026.07		33
deepseek-moe-16b-chat 2026.05		25.2
Minerva-7B-instruct-v1,0 2026.05		16.1
Olmo3 Base 2026.07		3.67
LMK token tuning 2026.07		2