Share your thoughts, 1 month free Claude Pro on usSee more

Long-context understanding on LongBench-E 1.0 (test)

44.56Qasper Score

Exact (Baseline)

Updated 24d ago

Evaluation Results

Method	Links
Exact (Baseline) 2025.02		44.56	50.65	69.14	60.39	21.3	19.52	75.33	81.14	43.17	22	99.67	50.9	35.22	51.77
Exact (Baseline) 2025.02		43.39	52.63	64.06	53.71	28.07	22.4	74.67	88.75	44.81	22.33	99	63.69	46.3	54.14
Exact (Baseline) 2025.02		42.87	48.54	52.05	38.6	31.31	22.07	71.67	91.85	42.36	20.37	98.13	49.62	42.73	50.17
BalanceKV 2025.02		40.14	43.17	64.46	58.06	22.26	20.32	73	80.68	41.07	22.33	92	44.95	32.43	48.84
Uniform 2025.02		39.28	43.82	64.82	57.84	23.1	19.5	73	81.63	39.7	22	92	44.97	32.19	48.76
Uniform 2025.02		37.18	37.15	47.49	37.82	27.56	21.06	68.67	90.48	36.13	20.33	96.26	45.72	37.09	46.38
Uniform 2025.02		37.07	41.69	61.11	49.96	29.18	22.67	71.67	87.89	40.34	22	84.33	58	42.57	49.88
BalanceKV 2025.02		37.02	41.96	61.74	50.9	29.26	22.64	71.67	88.1	41.14	23.67	87.67	58.64	43.63	50.62
SnapKV 2025.02		36.21	46.78	66.64	57.02	16.35	16.07	70.33	77.53	41.08	22	99.33	49.04	35.62	48.77
BalanceKV 2025.02		35.75	37.04	46.37	36.24	27.09	20.84	69	90.88	37.88	20.39	96.65	48.45	41.4	46.77
PyramidKV 2025.02		34.47	46.33	67.78	55.92	15.16	15.39	69.33	63.24	40.36	22.67	99.33	48.32	34.35	47.13
SnapKV 2025.02		33.91	42.55	49.09	36.13	20.48	17.67	62	91.7	40.23	20.33	98.86	46.7	39.86	46.12
PyramidKV 2025.02		33.86	39.75	47.12	35.96	20.03	17.78	63.67	90.76	40.21	20.4	98.97	45.25	39.51	45.64
SnapKV 2025.02		32.95	47.53	61.96	50.34	20.29	18.28	60.67	88.75	42.97	22.67	99.33	61.32	45.84	50.22
PyramidKV 2025.02		31.76	46.6	62.83	50	19.08	17.68	65	85.52	42.61	22	99.33	60.62	44.4	49.8
StreamingLLM 2025.02		20.73	32.62	49.93	42.39	21.63	18.69	59.67	74.96	29.57	11.67	63	46.16	32.25	38.71
StreamingLLM 2025.02		20.65	30.71	39.14	32.43	23.1	18.7	58	83.87	28.85	20.36	97.26	33.69	30.46	39.79
StreamingLLM 2025.02		20.12	34.35	51.84	48.23	19.09	17.1	61	51.14	28.52	23.33	41.33	39.19	26.54	35.52