Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Retrieval on Needle-in-a-Haystack

100Retrieval Accuracy

Qwen2.5-3B-Instruct (Teacher)

Updated 19h ago

Evaluation Results

Method	Links
Qwen2.5-3B-Instruct (Teacher) 2025.12		100
Hybrid student (25% softmax, 75% GDN) 2025.12		100
Qwen2.5-3B-Instruct (Teacher) 2025.12		100
Qwen2.5-3B-Instruct (Teacher) 2025.12		100
Qwen2.5-3B-Instruct (Teacher) 2025.12		100
USMoE 2025.03		100
Transformer 2026.06		100
SISA 2026.06		100
Transformer 2026.06		100
SISA 2026.06		100
Transformer 2026.06		100
SISA 2026.06		100
SISA 2026.06		100
SISA 2026.06		100
Mamba-3 2026.06		100
Hybrid student (25% softmax, 75% GDN) 2025.12		99.8
Hybrid student (25% softmax, 75% GDN) 2025.12		99.8
SISA 2026.06		99.5
Hybrid student (25% softmax, 75% GDN) 2025.12		99.4
Mamba-3 2026.06		99
Mamba-2 2026.06		96.5
Qwen2.5-3B-Instruct (Teacher) 2025.12		95.4
Mamba-3 2026.06		86.5
Mamba-2 2026.06		82.5
Hybrid student (25% softmax, 75% GDN) 2025.12		68.4
Mamba-3 2026.06		63.5
Original (TC) 2025.03		50
EC 2025.03		10
Mamba-2 2026.06		0