Share your thoughts, 1 month free Claude Pro on usSee more

Long-context language modeling on LongBench (test)

50.87Qasper Score

Full-context

Updated 25d ago

Evaluation Results

Method	Links
Full-context 2026.04		50.87	30.21	31.89	79.82	92.74	53.15	69.87	75.43	-	58.43	-	-	38.65	38.76	17.56	100	55.82	34.21	67.54	53.92	-
SnapKV 2026.04		50.23	29.34	30.56	78.45	92.12	52.45	68.98	66.54	-	56.78	-	-	36.23	37.98	16.12	99.5	54.05	33.12	63.56	52.89	-
StructKV 2026.04		50.15	30.12	31.34	79.43	92.66	53.01	68.72	74.56	-	57.67	-	-	37.42	38.56	16.67	100	55.19	34.12	64.89	53.67	-
FastKV 2026.04		50.12	29.67	30.78	78.89	92.12	52.34	68.45	73.12	-	57.23	-	-	36.45	38.34	15.98	99.5	54.49	33.56	63.23	52.12	-
StreamingLLM 2026.04		49.87	28.45	30.23	78.12	92.15	51.67	67.98	66.87	-	55.43	-	-	33.56	37.98	14.32	90.5	52.57	32.56	59.12	52.34	-
FastKV 2026.04		49.85	28.92	30.15	77.45	91.56	51.89	67.89	72.12	-	56.78	-	-	35.67	37.89	15.43	99.5	53.88	33.12	62.45	51.34	-
SnapKV 2026.04		49.76	28.65	30.23	77.23	91.56	52.12	68.12	65.67	-	55.98	-	-	35.12	37.56	15.34	99.5	53.46	32.87	63.12	52.45	-
StructKV 2026.04		49.65	29.87	31.23	79.12	92.45	52.78	68.56	73.34	-	56.34	-	-	36.78	38.34	15.83	100	54.62	33.15	64.21	52.24	-
StreamingLLM 2026.04		49.12	27.89	29.56	77.34	91.56	50.89	66.78	65.43	-	54.37	-	-	32.18	37.15	13.45	90.5	51.62	30.84	57.65	51.23	-
GemFilter 2026.04		49.12	27.87	29.34	77.45	91.89	48.78	66.12	55.67	-	51.23	-	-	34.87	36.98	12.12	96.5	51.01	29.89	57.12	51.23	-
GemFilter 2026.04		48.34	27.12	28.78	76.23	91.45	48.12	65.34	53.23	-	49.87	-	-	33.45	36.23	10.87	94.5	49.94	29.12	56.23	50.12	-
Full Cache 2026.06		45.37	25.38	27.2	72.5	91.64	43.57	62.88	56.43	-	55.8	-	-	31.26	-	9.41	99.5	49.2	30.22	55.97	45	35.12
K-VEC 2026.06		45.03	24.99	26.04	71.49	91.72	42.69	62.99	56.55	-	56.64	-	-	31.39	-	8.45	99.5	48.92	30.04	58.2	48.09	28.98
Ada-Pyramid 2026.06		44.57	25.26	25.84	70.5	91.62	42.37	62.96	56.52	-	53.73	-	-	31.97	-	7.67	99.5	48	28.76	54.89	44.15	27.75
Ada-SnapKV 2026.06		44.09	24.73	26.04	72.5	91.72	42.56	63.22	56.33	-	53.82	-	-	31.4	-	7.82	99.5	48.16	29.23	54.8	44.01	28.86
Pyramid 2026.06		43.66	24.3	25.68	68.5	91.27	41.96	63.13	55.85	-	54.1	-	-	31.3	-	7.73	99.5	47.7	29.62	55.06	44.22	27.27
SnapKV 2026.06		43.64	24.39	25.95	69	91.72	42.5	62.99	56.45	-	54.34	-	-	31.52	-	7.8	99.5	47.84	29.28	54.24	44.34	27.8
K-VEC 2026.06		42.81	24.86	24.92	67.47	91.62	42.67	62.9	55.68	-	56.07	-	-	31.39	-	8.45	99.5	48.12	30.04	57.9	47.39	26.24
PyramidInfer 2026.04		42.76	25.98	23.89	73.23	90.56	49.87	66.23	65.78	-	46.54	-	-	28.23	31.87	9.12	88.5	48.06	29.87	49.12	47.34	-
Full 2026.03		41.54	25.71	27.73	76	88.59	47.51	61.48	62.68	29.07	52.88	49.37	39.01	28.58	35.07	6	98.5	48.11	-	-	-	-
MixedDimKV 2026.03		41.5	25.33	27.59	75	88.75	47.81	61.44	62.3	29.62	53.05	48.49	38.7	28.16	30.67	6	98	47.65	-	-	-	-
Ada-Pyramid 2026.06		40.63	24.12	24.61	69	91.79	42.55	62.28	54.49	-	53.03	-	-	30.26	-	7.95	99.5	46.98	28.04	54.71	43.39	25.35
K-VEC 2026.06		40.38	23.94	22.95	62.74	91.89	41.95	60.87	54.9	-	53.53	-	-	31.24	-	8.05	99.5	46.7	26.33	57.39	47.14	24.35
Ada-SnapKV 2026.06		40.16	24.39	24.85	69	92.34	42.05	63.43	55.32	-	52.44	-	-	31.1	-	7.98	99.5	47.15	29.07	53.9	43.05	25.75
SnapKV 2026.06		40.01	23.77	24.19	64	92.05	41.57	63.21	55.05	-	53.15	-	-	31.32	-	8.01	99.5	46.76	29.22	54.47	43.63	25.04
Pyramid 2026.06		39.89	23.73	24.28	66	91.07	41.42	61.99	53.44	-	52.86	-	-	31.22	-	8.39	99.5	46.59	28.71	54	44.2	24.74
K-VEC 2026.06		35.96	23.47	21.88	52.89	90.35	41.25	59.35	51.25	-	50.4	-	-	28.75	-	7.85	99.5	44.57	25.96	54.79	47.14	22.35
Ada-Pyramid 2026.06		35.06	23.36	22.67	63	90.72	41.21	61.47	53.09	-	52.28	-	-	28.76	-	7.75	99.5	45.23	25.12	54.66	41.89	23.14
H2O 2026.03		34.24	21.61	25.89	72.5	87.69	40.96	53.94	47.76	25.93	42.34	46.22	33.65	22.8	26.9	6.5	88	42.31	-	-	-	-
Ada-SnapKV 2026.06		33.39	23.85	22.88	63.5	91.57	40.94	61.95	54.33	-	51.44	-	-	29.54	-	8	99.5	45.44	26.11	54.94	42.15	23.01
Pyramid 2026.03		33.36	23.2	23.25	72.5	89.56	45.14	56.9	55.62	27.6	52.23	48.48	38.11	27.1	23.64	5.5	96	44.89	-	-	-	-
SnapKV 2026.03		33.18	23.12	23.21	73	88.79	44.88	58.05	58.12	27.72	51.59	47.58	37.6	27.73	23.98	5	96	44.97	-	-	-	-
Pyramid 2026.06		33.15	23.37	22.33	56.5	91.19	41.28	60.36	51.18	-	51.44	-	-	28.62	-	6.97	99.5	44.47	25.53	55.03	42.42	22.57
SnapKV 2026.06		32.49	23.54	22.55	53.5	91.1	40.95	60.67	53.39	-	51.62	-	-	28.94	-	7.48	99.5	44.48	26.02	54.4	42.77	22.83
Full 2026.03		32.3	23.22	26.67	74	90.48	42.29	59.13	54.02	25.56	39.71	43.56	35.49	21.14	28.58	4.8	69.75	41.92	-	-	-	-
MixedDimKV 2026.03		31.39	22.95	26.54	73	90.61	42.4	59.11	54.84	25.26	39.6	43.82	35.46	21.59	27.07	5.02	69.5	41.76	-	-	-	-
SLM 2026.06		30.22	21.08	25.72	63.5	88.87	42.28	61.3	53.4	-	37.06	-	-	25.24	-	6.98	89	42.58	24.97	46.57	39.14	26.01
SLM 2026.06		25.78	21.26	23.57	57.5	87.86	41.44	60.85	51.02	-	34.19	-	-	24.93	-	6.98	96.5	41.37	25.51	45.01	35.91	23.61
Ada-Pyramid 2026.06		25.61	23.18	20.38	51.5	90.76	40.62	59.3	50.88	-	49.3	-	-	27.83	-	6.92	99	42.96	27.07	53.02	41.29	20.7
Pyramid 2026.06		24.69	22.38	19.74	44.5	88.81	40.3	57.25	49.9	-	47.74	-	-	27.23	-	7.22	99.5	41.78	25.7	52.87	40.57	20.02
SnapKV 2026.06		24.45	22.36	19.55	45.5	89.2	40.62	58.45	49.9	-	48.03	-	-	28.19	-	6.97	99.5	42.03	25.54	53.31	40.75	20.13
Ada-SnapKV 2026.06		24.41	23.21	20.28	50.5	89.49	40.71	58.74	52.4	-	49.95	-	-	28.55	-	7.45	99	42.81	24.9	53.15	41.73	20.54
SLM 2026.06		23.79	20.68	20.67	46	87.11	40.82	59.89	49.19	-	31.8	-	-	25.55	-	7.2	99.5	39.76	22.71	43.43	36.55	21.29
H2O 2026.04		22.12	26.89	29.87	74.87	93.12	49.65	66.23	62.45	-	35.23	-	-	20.87	36.56	11.89	85.5	43.56	28.98	26.54	26.12	-
SLM 2026.06		20.87	21.09	18.48	40.5	84.41	38.82	57.02	47.29	-	31.72	-	-	24.54	-	8	99.5	38.43	22.24	44.02	37.55	18.76
Pyramid 2026.03		20.51	22.68	22.83	71	90.48	39.86	58.64	54.06	23.99	36.06	42.47	31.34	20.28	21.36	5.83	69.25	39.42	-	-	-	-
SnapKV 2026.03		20.2	22	22.83	71	90.86	40.03	60.16	55.91	23.36	37.36	42.37	33.18	19.96	21.76	5.83	69.5	39.77	-	-	-	-
H2O 2026.04		19.34	27.45	30.65	74.23	92.65	48.98	64.12	61.23	-	32.67	-	-	16.43	36.87	11.23	82.5	42	27.56	23.45	22.56	-
H2O 2026.03		18.56	21.48	24.57	60.5	88.7	38.55	58.52	52.73	23.09	29.02	36.01	28.07	15.7	23.31	5.37	68.04	37.01	-	-	-	-
MoQAE 2025.06		9.79	21.23	3.47	66	87.89	41.37	66.53	59.94	-	-	-	-	-	-	-	-	-	-	-	-	-
CQ-4c8b 2025.06		9.58	20.87	1.93	66	87.72	41.13	66.57	59.75	-	-	-	-	-	-	-	-	-	-	-	-	-
FP16 2025.06		9.52	21.28	3.51	66	87.72	41.69	66.66	59.82	-	-	-	-	-	-	-	-	-	-	-	-	-
KIVI-2b 2025.06		9.26	20.53	0.97	66	87.42	42.61	66.22	59.67	-	-	-	-	-	-	-	-	-	-	-	-	-
MiKV 2025.06		9.14	20.63	0.85	65.88	87.21	41.44	66.18	59.55	-	-	-	-	-	-	-	-	-	-	-	-	-