Share your thoughts, 1 month free Claude Pro on usSee more

Long-context language understanding on LongBench (Comprehensive 16-Task Subset)

30.46NtrvQA

Full Cache

Updated 1d ago

Evaluation Results

Method
Full Cache 2026.05	30.46	47.29	55.03	58.65	51.61	33.03	35.03	24.97	27.04	29	84.81	39.29	10.15	100	51.43	44.19	45.12
CaM 2026.05	29.29	30.05	39.26	53.79	39	27.6	28.51	22.76	23.42	22	82.55	39.45	8.79	96.5	46.94	44.19	39.63
GRKV 2026.05	29.24	32.87	39.86	53.47	42.68	26.13	29.27	23.04	23.56	50	81.83	41.68	10.05	97	53.55	46.15	42.52
CriticalKV 2026.05	28.94	31.35	39.22	53.57	41.79	25.94	29.27	23.03	23.4	49	80.58	41.01	9.55	96.5	52.89	45.99	42
Full Cache 2026.05	28.38	40.72	51.68	49	36.21	27.81	34.23	25.41	26.93	55.75	84.93	20.94	5.05	98	46.92	53.86	42.86
SnapKV 2026.05	27.84	27.19	34.18	54.98	37.9	24.48	27.76	21.86	22.72	38	83.01	40.94	7.55	89.5	51.73	44.31	39.62
GRKV 2026.05	27.73	28.3	34.36	54.45	36.85	25.99	27.98	22.04	23.11	41.5	84.06	41.17	8.55	92	51.47	44.86	40.28
D2O 2026.05	27.18	26.55	31.28	49.36	31.41	22.61	25.97	21.18	21.95	36	82.81	39.93	10	74.5	50.85	45.46	37.32
CaM 2026.05	26.16	26.61	35.54	54.42	37.46	26	27.03	21.87	22.59	20.07	84.78	39.56	8.68	86.5	46.53	42.72	37.91
AsymKV 2026.05	25.6	30.06	36.48	49.08	33.52	19.46	27.92	22.29	24.4	17.5	89.24	26.99	5.61	63.5	49.1	44.95	35.36
D2O 2026.05	25.11	19.71	29.16	48.41	25.52	19.83	24.91	19.93	20.88	33	82.41	38.91	7.04	66.5	50.08	45.05	34.78
AsymKV 2026.05	24.57	24.1	35.83	42.84	28.98	16.77	28.72	21.64	23.78	45.82	85.61	24.71	1.61	60.17	45.81	48.68	34.98
GRKV 2026.05	23.2	25.84	43.02	44.07	32.92	19.64	29.07	23.08	23.69	47	88.03	24.3	4.88	93.5	50.49	52.02	39.05
AsymKV 2026.05	23.06	24.27	37.25	45.63	26.43	18.94	28.8	21.58	24.07	51	85.36	23.25	2.43	54.5	45.06	49.23	35.05
CaM 2026.05	22.93	24.64	41.2	45.48	33.7	22.26	28.27	22.86	23.42	35.07	85.44	11.63	4.25	90	48.95	50.17	36.89
CriticalKV 2026.05	22.88	25.68	42.55	42.53	-	18.68	28.76	23.02	23.22	47.25	86.99	23.14	4.65	93	49.49	51.65	38.47
AsymKV 2026.05	22.76	29.76	34.17	43.08	34.27	16.6	27.21	22.12	24.24	7	83.87	21.86	3.83	45.5	51.92	47.17	32.21
CaM 2026.05	19.5	18.3	35.13	40.35	21.94	20.41	27.57	21.88	22.81	36.5	84.68	14.04	2.47	87.17	47.69	50.85	34.46
D2O 2026.05	19.28	15.96	35.13	36.64	25.84	16.48	26.28	21.13	21.76	41.5	87.08	26.98	4.43	57.5	50.61	51.26	33.62
GRKV 2026.05	18.7	19.45	37.81	39.5	27.82	21.44	28.03	22.15	23	44	87.17	23.13	4.65	90	52.51	52.54	36.99
SnapKV 2026.05	18.06	18.46	36.68	39.7	25.67	20.95	27.83	21.68	22.62	43.75	86.42	23.42	4.55	89.5	51.47	52.6	36.46
D2O 2026.05	16.28	10.77	34.09	35.27	22.61	14.74	25.59	19.93	20.92	35.75	86.47	28.09	5.26	55	50.65	52.16	32.1