Share your thoughts, 1 month free Claude Pro on usSee more

Long-context language understanding on LongBench (Mistral-8B-Instruct)

30.21NrtvQA

StructKV

Updated 9d ago

Evaluation Results

Method	Links
StructKV 2026.04		30.21	46.88	51.56	61.43	57.55	38.23	32.1	24.02	25.98	74.5	92.04	45.37	10	100	65.02	56.5	50.68
StructKV 2026.04		30.06	46.54	51.33	62.36	56.74	38.64	31.12	23.99	25.86	74.5	92.04	45.47	10	100	65.02	56.15	50.61
GemFilter 2026.04		28.06	43.81	50.56	63.46	59.43	39.25	30.59	23.31	22.03	70.5	90.84	43.26	5.5	97	27.56	46.03	46.32
H2O 2026.04		27.75	40.61	44.04	52.19	48.78	30.68	29.4	23.14	25.35	72	90.66	43.26	7.5	70	61.99	59.91	45.45
H2O 2026.04		27.28	43.31	44.57	53.41	49.93	30.88	30.47	23.55	25.87	72	91.16	44.04	7.5	71	65.02	66.47	46.65
SnapKV 2026.04		26.82	43.55	49.53	60.18	51.72	36.38	26.94	23.55	21.27	72	92.04	44.01	10	100	65.02	66.47	49.34
SnapKV 2026.04		26.72	45.34	49.83	61.11	52.07	36.58	29.36	23.99	23.65	74.5	92.04	45.37	9	100	65.55	65.65	50.05
FastKV 2026.04		25.9	46.05	51.29	61.24	50.21	36.15	28.65	23.84	22.57	76	92.04	46.2	10	100	64.68	64.68	49.97
GemFilter 2026.04		25.7	37.97	39.73	60.62	56.04	37.22	28.65	21.18	19.57	63	89.36	42.79	6.5	85.5	59.96	41.79	44.72
FastKV 2026.04		25.31	43.71	50.11	61.31	50.85	36.73	26.15	23.45	20.54	75	92.04	45.1	10	100	62.99	65	49.27
PyramidInfer 2026.04		24.92	37.6	40.34	48.92	40.84	22.26	27.23	22.99	22.25	68.5	91.06	45.02	7.5	97	66.91	64.74	45.51
Full-context 2026.04		24.55	47.99	52.21	60.43	52.64	35.28	32.36	24.16	26.64	74.5	92.04	45.94	9	100	67.06	67.19	50.75
StreamingLLM 2026.04		23.25	35.91	28.26	51.46	41.13	24.08	26.88	21.38	21.43	67.5	91.88	44.65	5	72	63.17	63.21	42.57
StreamingLLM 2026.04		22.52	33.02	27.69	48.68	37.66	21.26	23.55	21.09	18.36	61.5	91.38	43.45	6	63	62.84	62.55	40.28