Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Understanding on LongBench LLaMA-3.1-8B-Instruct (test)

32.8NrtvQA

StructKV

Updated 9d ago

Evaluation Results

Method	Links
StructKV 2026.04		32.8	44.5	55.25	55.8	46.9	30.85	32.1	25.35	25.9	73.65	92.1	43.25	8.85	99.5	63.2	56.8	48.97
StructKV 2026.04		32.45	41.88	55.12	55.2	46.85	30.55	29.6	25.15	24.6	73.2	92.45	43	8.6	99.5	62.4	56.15	48.61
SnapKV 2026.04		31.51	40.18	41	55.83	44.3	30.92	28.88	24.58	22.57	70.5	91.28	42.64	7.93	99.5	62.32	56.71	46.92
SnapKV 2026.04		31.15	44.11	54.55	55.47	45.14	31.14	31.22	24.94	24.09	70.5	91.9	42.23	7.96	99.5	63.55	57.91	48.46
FastKV 2026.04		30.54	40.75	54.57	54.33	46.3	30.55	28.15	24.29	21.91	73	92.38	42.96	7.36	99.5	60.11	54.79	47.59
FastKV 2026.04		30.26	43.7	54.83	54.39	46.42	30.42	30.28	24.88	22.93	73.5	91.97	43.07	7.61	99.5	61.61	55.67	48.19
Full-context 2026.04		30.21	45.53	55.01	56.01	46.65	31.28	35.13	25.28	27.25	73	91.64	43.8	8.91	99.5	63.38	56.64	49.33
StreamingLLM 2026.04		27.99	28.89	34.42	51.21	41.03	24.43	28.34	22.08	22.16	64	89.96	42.27	8.66	85	62.02	56.85	43.08
StreamingLLM 2026.04		26.05	26.29	33.68	48.07	37.93	25.03	25.37	21.49	19.97	58.5	88.46	42.61	7.91	88	61.07	54.94	41.59
GemFilter 2026.04		26.02	30.74	47.3	37.97	55.97	20.64	31.19	20.92	21.06	61.5	92.75	40.92	6.18	97	31.99	41.12	41.42
GemFilter 2026.04		24.36	21.07	39.73	25.99	43.92	35.78	28.94	21.42	17.62	61	91.53	40.88	4.76	85	59.95	44.38	40.4
PyramidInfer 2026.04		12.1	35.54	37.58	17.66	33.71	6.56	27.84	21.62	21.04	64	91.89	43.31	3.65	70.08	64.92	57.9	38.09
H2O 2026.04		9.8	41.09	41	19.2	33.92	6.83	32.26	21.5	25.62	66	91.27	42.41	4.31	68.25	62.02	54.08	38.72
H2O 2026.04		9.32	34.96	37.08	19.65	32.65	6.91	31.29	21.01	25.59	66.5	91.03	42.16	3.41	65.75	58.27	51.48	37.32