Share your thoughts, 1 month free Claude Pro on usSee more

Long-context language modeling on LongBench (test)

50.87Qasper Score

Full-context

Updated 9d ago

Evaluation Results

Method	Links
Full-context 2026.04		50.87	30.21	31.89	79.82	92.74	53.15	69.87	75.43	-	58.43	-	-	38.65	38.76	17.56	100	55.82	34.21	67.54	53.92
SnapKV 2026.04		50.23	29.34	30.56	78.45	92.12	52.45	68.98	66.54	-	56.78	-	-	36.23	37.98	16.12	99.5	54.05	33.12	63.56	52.89
StructKV 2026.04		50.15	30.12	31.34	79.43	92.66	53.01	68.72	74.56	-	57.67	-	-	37.42	38.56	16.67	100	55.19	34.12	64.89	53.67
FastKV 2026.04		50.12	29.67	30.78	78.89	92.12	52.34	68.45	73.12	-	57.23	-	-	36.45	38.34	15.98	99.5	54.49	33.56	63.23	52.12
StreamingLLM 2026.04		49.87	28.45	30.23	78.12	92.15	51.67	67.98	66.87	-	55.43	-	-	33.56	37.98	14.32	90.5	52.57	32.56	59.12	52.34
FastKV 2026.04		49.85	28.92	30.15	77.45	91.56	51.89	67.89	72.12	-	56.78	-	-	35.67	37.89	15.43	99.5	53.88	33.12	62.45	51.34
SnapKV 2026.04		49.76	28.65	30.23	77.23	91.56	52.12	68.12	65.67	-	55.98	-	-	35.12	37.56	15.34	99.5	53.46	32.87	63.12	52.45
StructKV 2026.04		49.65	29.87	31.23	79.12	92.45	52.78	68.56	73.34	-	56.34	-	-	36.78	38.34	15.83	100	54.62	33.15	64.21	52.24
StreamingLLM 2026.04		49.12	27.89	29.56	77.34	91.56	50.89	66.78	65.43	-	54.37	-	-	32.18	37.15	13.45	90.5	51.62	30.84	57.65	51.23
GemFilter 2026.04		49.12	27.87	29.34	77.45	91.89	48.78	66.12	55.67	-	51.23	-	-	34.87	36.98	12.12	96.5	51.01	29.89	57.12	51.23
GemFilter 2026.04		48.34	27.12	28.78	76.23	91.45	48.12	65.34	53.23	-	49.87	-	-	33.45	36.23	10.87	94.5	49.94	29.12	56.23	50.12
PyramidInfer 2026.04		42.76	25.98	23.89	73.23	90.56	49.87	66.23	65.78	-	46.54	-	-	28.23	31.87	9.12	88.5	48.06	29.87	49.12	47.34
Full 2026.03		41.54	25.71	27.73	76	88.59	47.51	61.48	62.68	29.07	52.88	49.37	39.01	28.58	35.07	6	98.5	48.11	-	-	-
MixedDimKV 2026.03		41.5	25.33	27.59	75	88.75	47.81	61.44	62.3	29.62	53.05	48.49	38.7	28.16	30.67	6	98	47.65	-	-	-
H2O 2026.03		34.24	21.61	25.89	72.5	87.69	40.96	53.94	47.76	25.93	42.34	46.22	33.65	22.8	26.9	6.5	88	42.31	-	-	-
Pyramid 2026.03		33.36	23.2	23.25	72.5	89.56	45.14	56.9	55.62	27.6	52.23	48.48	38.11	27.1	23.64	5.5	96	44.89	-	-	-
SnapKV 2026.03		33.18	23.12	23.21	73	88.79	44.88	58.05	58.12	27.72	51.59	47.58	37.6	27.73	23.98	5	96	44.97	-	-	-
Full 2026.03		32.3	23.22	26.67	74	90.48	42.29	59.13	54.02	25.56	39.71	43.56	35.49	21.14	28.58	4.8	69.75	41.92	-	-	-
MixedDimKV 2026.03		31.39	22.95	26.54	73	90.61	42.4	59.11	54.84	25.26	39.6	43.82	35.46	21.59	27.07	5.02	69.5	41.76	-	-	-
H2O 2026.04		22.12	26.89	29.87	74.87	93.12	49.65	66.23	62.45	-	35.23	-	-	20.87	36.56	11.89	85.5	43.56	28.98	26.54	26.12
Pyramid 2026.03		20.51	22.68	22.83	71	90.48	39.86	58.64	54.06	23.99	36.06	42.47	31.34	20.28	21.36	5.83	69.25	39.42	-	-	-
SnapKV 2026.03		20.2	22	22.83	71	90.86	40.03	60.16	55.91	23.36	37.36	42.37	33.18	19.96	21.76	5.83	69.5	39.77	-	-	-
H2O 2026.04		19.34	27.45	30.65	74.23	92.65	48.98	64.12	61.23	-	32.67	-	-	16.43	36.87	11.23	82.5	42	27.56	23.45	22.56
H2O 2026.03		18.56	21.48	24.57	60.5	88.7	38.55	58.52	52.73	23.09	29.02	36.01	28.07	15.7	23.31	5.37	68.04	37.01	-	-	-
MoQAE 2025.06		9.79	21.23	3.47	66	87.89	41.37	66.53	59.94	-	-	-	-	-	-	-	-	-	-	-	-
CQ-4c8b 2025.06		9.58	20.87	1.93	66	87.72	41.13	66.57	59.75	-	-	-	-	-	-	-	-	-	-	-	-
FP16 2025.06		9.52	21.28	3.51	66	87.72	41.69	66.66	59.82	-	-	-	-	-	-	-	-	-	-	-	-
KIVI-2b 2025.06		9.26	20.53	0.97	66	87.42	42.61	66.22	59.67	-	-	-	-	-	-	-	-	-	-	-	-
MiKV 2025.06		9.14	20.63	0.85	65.88	87.21	41.44	66.18	59.55	-	-	-	-	-	-	-	-	-	-	-	-