Share your thoughts, 1 month free Claude Pro on usSee more

Long-context language modeling evaluation on HELMET

0Average Sparsity

FullAttention

Updated 1mo ago

Evaluation Results

Method	Links
FullAttention 2026.02		0	50.33	48.87	46.11	43.83	41.07	46.04
FullAttention 2026.02		0	67.09	67.42	66.78	65.73	62.8	65.96
FullAttention 2026.02		0	61.17	58.67	56.71	55.81	49.74	56.42
FullAttention 2026.02		0	54.75	50.66	48.4	44.98	39.59	47.68
FlexPrefill 2026.02		37.88	53.18	49.05	46.48	43.01	35.52	45.45
FlexPrefill 2026.02		38.63	60.73	58.15	56.22	55.15	49.87	56.02
FlexPrefill 2026.02		40.16	50.87	50.01	46	43.87	40.92	46.33
XAttention 2026.02		41.96	66.51	67.22	66.38	63.75	61.17	65.01
RRAttention 2026.02		43.01	66.96	67.63	66.55	64.46	61.66	65.45
XAttention 2026.02		47.35	53.9	49.77	46.96	44.49	38.09	46.64
XAttention 2026.02		47.5	60.79	58.1	55.57	54.78	49.45	55.74
RRAttention 2026.02		47.56	54.22	50.02	47.92	45.17	38.51	47.17
FlexPrefill 2026.02		48.22	66.15	65.61	64.97	63.2	60.8	64.15
RRAttention 2026.02		48.68	61.22	58.21	56.07	55.33	50.37	56.24
XAttention 2026.02		50.8	50.41	48.51	44.73	43.69	41.17	45.7
RRAttention 2026.02		51.86	50.95	49.45	47.13	44.12	41.06	46.54
XAttention 2026.02		55.2	66.78	67.19	65.25	63.12	60.04	64.48
RRAttention 2026.02		56.26	67.01	66.6	65.71	62.97	60.7	64.6
FlexPrefill 2026.02		58.42	64.08	64.11	63.15	61.64	59.3	62.46
XAttention 2026.02		59.15	52.53	47.72	45.72	43.35	37.04	45.27
RRAttention 2026.02		59.53	53.37	49.31	47.35	44.62	38.48	46.63
XAttention 2026.02		59.88	59.92	57.15	55.54	53.66	47.82	54.82
RRAttention 2026.02		61.02	60.95	58.43	55.86	54.27	48.85	55.67
FlexPrefill 2026.02		63.21	44.11	40.59	40.6	36.39	29.66	38.27
XAttention 2026.02		63.85	50.28	47.79	44.17	43.31	40.19	45.15
RRAttention 2026.02		64.05	50.79	48.58	46.48	43.89	40.77	46.1
FlexPrefill 2026.02		65.98	44.99	44.51	43.27	41.31	38.73	42.56
FlexPrefill 2026.02		66.07	56.02	54.48	53.01	50.77	48.42	52.54