Share your thoughts, 1 month free Claude Pro on usSee more

Recall-intensive Retrieval on SWDE, SQuAD, FDA, TQA, NQ, and Drop Suite

76.85Performance on SWDE

Qwen3-8B

Updated 6d ago

Evaluation Results

Method	Links
Qwen3-8B 2025.10		76.85	59.89	73.48	72.63	45.01	36.61	60.75
Qwen2.5-7B 2025.10		75.45	54.28	82.38	77.49	48.43	38.28	62.72
Llama-3-8B 2025.10		72.07	54.22	82.92	74.17	43.24	33.88	60.08
Zamba-7B 2025.10		71.7	50.69	76.02	70.56	42.1	25.78	56.14
StripedHyena-7B 2025.10		69.82	53.81	75.3	71.86	42.41	32.34	57.59
NHA-Llama-3-8B 2025.10		67.67	55.16	73.84	73.05	42.1	34.02	57.64
Mistral-7B 2025.10		65.32	54.01	75.3	73.16	42.76	27.55	56.35
Zamba2-7B 2025.10		64.48	50.15	71.48	71.86	40.13	29.23	54.56
FalconMamba-7B 2025.10		59.51	49.31	48.32	75.06	37.85	30.09	50.02
RWKV-6-World-7B 2025.10		51.55	46.62	56.86	69.85	37.03	28.7	48.44
Oryx-TM 2026.05		50.1	41.7	57.6	63.9	28.3	22.5	44
NHA-Qwen2.5-7B 2025.10		44.61	47.26	62.58	71.98	37.47	37.47	50.23
Oryx-TG 2026.05		44.5	41.2	46.4	64.8	27.8	21.7	41.1
Mamba-7B 2025.10		43.3	46.93	37.06	71.56	33.39	25.3	42.92
Baseline 2026.05		42.3	42.8	55.2	65.9	26.7	22.9	42.6
Gated DeltaNet-H2 2025.10		38.2	40.4	50.7	63.3	24.8	23.3	40.1
Nirvana 2025.10		37.8	41	51.1	62.8	24.8	22.9	40.1
Gated DeltaNet-H1 2025.10		35.6	39.7	52	60.1	24.6	22.2	39
Nirvana-noTrigger 2025.10		35.1	39.8	50.5	60	22.3	21.7	38.2
Samba 2025.10		33	39.2	50.5	57.7	23.5	20.2	37.3
Transformer++ 2025.10		29.5	38	52.2	58.3	22.5	21.6	37
Oryx-TG 2026.05		29.3	37.9	18.2	64	23.5	22	32.5
Baseline 2026.05		27.4	36.4	24.2	63.3	23.8	21.6	32.8
Gated DeltaNet 2025.10		25.4	34.8	23.7	60	20	19.8	30.6
Oryx-TM 2026.05		22.2	36.4	19.4	63.7	21.8	22.3	31
Baseline 2026.05		20.6	36.1	21	61.7	22.4	20.3	30.3
Mamba2 2025.10		19.1	33.6	25.3	61	20.8	19.2	29.8
DeltaNet 2025.10		17.9	30.9	18.4	53.9	17.3	18.6	26.2
RetNet 2025.10		14	28.5	7	54.4	16.2	17.3	22.9
Mamba 2025.10		9.8	25.8	3.7	54.3	14.9	17.4	21
HGRN2 2025.10		8.3	25.3	4.8	51.2	14.2	16.9	20.1