Our new X account is live! Follow @wizwand_team for updates

Length Extrapolation on RULER (8K-256K Contexts)

92.88Performance @ 8K Context

Llama-3.1-8B-Instruct

Updated 4d ago

Evaluation Results

Method	Links
Llama-3.1-8B-Instruct 2026.01		92.88	92.83	89.46	70.79	80.12	72.34	83.47	-
Elastic Attention 2026.01		92.82	92	87.8	68.23	78.87	68.51	81.82	0.72
Qwen3-8B 2026.01		89.69	85.62	63.23	82.39	65.84	66.71	75.74	-
InfLLM-V2 2026.01		89.3	80.93	60.98	35.9	32.29	47.27	59.1	-
Qwen3-4B 2026.01		87.49	86.82	60.05	70.98	53.19	43.27	66	-
PruLong 2026.01		87.2	82.69	61.64	73.05	59.74	58.82	69.9	0.7
DuoAttention 2026.01		87.2	77.85	66.99	40.13	57.45	42.92	62.92	0.7
Elastic Attention 2026.01		86.62	82.81	64.55	77.41	61.17	61.75	71.74	0.65
Elastic Attention 2026.01		86.56	85.38	56.88	69.42	49.48	43.47	63.27	0.67
Elastic Attention 2026.01		85.07	85.12	65.08	82.34	64.57	63.41	73.87	0.76
PruLong 2026.01		83.54	69.35	56.83	30.88	33.74	21.64	48.82	0.7
Elastic Attention 2026.01		83.35	79.24	50.79	67.03	47.83	47.32	61.81	0.66
DuoAttention 2026.01		81.03	78.85	56.61	70.1	54.68	56.28	65.94	0.7
Elastic Attention 2026.01		80.2	83.42	-	56.3	68.16	56.47	72.85	0.65
InfLLM-V2 2026.01		78.59	74.4	43.39	44.94	28.57	27.01	51.02	-
PruLong 2026.01		77.03	73.99	51.32	60.98	42.7	48.76	58.38	0.7
InfLLM-V2 2026.01		77.01	68.53	25.93	53.47	34.96	32.95	52.58	-
DuoAttention 2026.01		76.82	75.17	50.53	61.42	45.86	45.46	58.3	0.7