Share your thoughts, 1 month free Claude Pro on usSee more

Recall-intensive Retrieval on SWDE, SQuAD, FDA, TQA, NQ, and Drop Suite

76.85Performance on SWDE

Qwen3-8B

Updated 3d ago

Evaluation Results

Method	Links
Qwen3-8B 2025.10		76.85	59.89	73.48	72.63	45.01	36.61	60.75
Qwen2.5-7B 2025.10		75.45	54.28	82.38	77.49	48.43	38.28	62.72
Llama-3-8B 2025.10		72.07	54.22	82.92	74.17	43.24	33.88	60.08
Zamba-7B 2025.10		71.7	50.69	76.02	70.56	42.1	25.78	56.14
StripedHyena-7B 2025.10		69.82	53.81	75.3	71.86	42.41	32.34	57.59
NHA-Llama-3-8B 2025.10		67.67	55.16	73.84	73.05	42.1	34.02	57.64
Mistral-7B 2025.10		65.32	54.01	75.3	73.16	42.76	27.55	56.35
Zamba2-7B 2025.10		64.48	50.15	71.48	71.86	40.13	29.23	54.56
FalconMamba-7B 2025.10		59.51	49.31	48.32	75.06	37.85	30.09	50.02
RWKV-6-World-7B 2025.10		51.55	46.62	56.86	69.85	37.03	28.7	48.44
NHA-Qwen2.5-7B 2025.10		44.61	47.26	62.58	71.98	37.47	37.47	50.23
Mamba-7B 2025.10		43.3	46.93	37.06	71.56	33.39	25.3	42.92
Gated DeltaNet-H2 2025.10		38.2	40.4	50.7	63.3	24.8	23.3	40.1
Nirvana 2025.10		37.8	41	51.1	62.8	24.8	22.9	40.1
Gated DeltaNet-H1 2025.10		35.6	39.7	52	60.1	24.6	22.2	39
Nirvana-noTrigger 2025.10		35.1	39.8	50.5	60	22.3	21.7	38.2
Samba 2025.10		33	39.2	50.5	57.7	23.5	20.2	37.3
Transformer++ 2025.10		29.5	38	52.2	58.3	22.5	21.6	37
Gated DeltaNet 2025.10		25.4	34.8	23.7	60	20	19.8	30.6
Mamba2 2025.10		19.1	33.6	25.3	61	20.8	19.2	29.8
DeltaNet 2025.10		17.9	30.9	18.4	53.9	17.3	18.6	26.2
RetNet 2025.10		14	28.5	7	54.4	16.2	17.3	22.9
Mamba 2025.10		9.8	25.8	3.7	54.3	14.9	17.4	21
HGRN2 2025.10		8.3	25.3	4.8	51.2	14.2	16.9	20.1