Share your thoughts, 1 month free Claude Pro on usSee more

Long-context language understanding on RULER 16K 1.0 (test)

89.28CWE Score

Full Cache

Updated 1d ago

Evaluation Results

Method	Links
Full Cache 2026.05		89.28	90.33	99.6	100	99	98.9	98.9	100	100	100	80.8	57.2	99.72	93.36	-
Full Cache 2026.05		82.64	88.07	97.6	95	77.6	88.5	88.65	94.4	96.4	99.6	72	49.8	96.04	86.64	-
CriticalKV w. GRKV 2026.05		59.84	79.33	13.6	2.8	0	10.15	9.75	39.2	9.6	2.4	28	31.6	14	23.1	10
CriticalKV 2026.05		59.1	78.2	13	2.4	0	10.05	9.45	36.8	9.2	2.4	28.2	31.4	13.24	22.57	0
SnapKV w. GRKV 2026.05		47.32	83.33	10.8	2	0.4	9.8	9.7	42.2	4.4	2.4	24.6	30.4	14.32	21.67	10
SnapKV 2026.05		45.98	82.8	10.4	1.8	0.4	9.6	9.5	40.8	3.4	2.4	24.2	31.8	12.32	21.18	0
CriticalKV w. D2O 2026.05		30.74	71.33	9.8	0.2	0	8.7	9.5	40.4	5.8	2.4	22	29	4.56	18.03	2
SnapKV w. D2O 2026.05		22.78	73.13	8.2	0	0	8.7	9.2	43.2	4	2.4	21.6	28.6	2.68	17.27	2
SnapKV w. CaM 2026.05		10.32	64.73	10.2	1.4	0	9.05	9.2	5.4	4	2.4	25.4	27.4	1.44	13.15	2
CriticalKV 2026.05		9.72	58.73	50.2	5.6	1.6	45.1	46.3	93.6	94.4	2.6	23.8	30.4	68.72	40.83	0
CriticalKV w. GRKV 2026.05		8.8	64.93	50	5.8	1.6	45.6	46.65	93.6	95.4	2.6	24	31	69.6	41.51	8
SnapKV 2026.05		3.24	59	23.2	4	1.4	17	15.1	78.4	54.2	2.4	21	28	49.72	27.44	0
CriticalKV w. CaM 2026.05		2.06	19.8	11.8	0.2	0	9.15	9.2	0	5.6	2.4	26	26.8	0	8.69	0
SnapKV w. GRKV 2026.05		1.78	66.4	25.6	4.4	1.4	17.6	15.8	82.4	58.2	2.4	20.2	27.8	54.24	29.09	8
CriticalKV w. D2O 2026.05		0.28	53.8	36.4	1	0	36.95	32.45	93.8	80.2	2.4	15	26.2	52.28	33.14	1
SnapKV w. CaM 2026.05		0.12	56.2	17.6	2.2	1.6	13.95	13.2	41.4	31.2	2.4	23	28.2	5.44	18.19	3
CriticalKV w. CaM 2026.05		0.1	57.87	34.2	2.2	1.2	28.65	28.05	50.6	71	2.6	25.6	25.8	6.48	25.72	1
SnapKV w. D2O 2026.05		0.02	57.8	16.4	0.4	0.2	12.7	11.25	79.8	47.6	2.4	14.6	23.8	36.6	23.35	1