Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Evaluation on Ruler (test)

100S-NIAH-1

FullKV

Updated 1mo ago

Evaluation Results

Method
FullKV 2026.03	100	100	100	99.6	99.6	99.6	99.9	99.75	83.98	90.67	100	97.55
H2O 2026.03	100	99.8	22	99.8	84.8	32.4	99.5	89.9	46.74	93	99.92	78.9
PyramidKV 2026.03	100	100	24.8	99.6	91	51.2	99.9	99.55	57.4	92.87	100	83.3
SnapKV 2026.03	100	100	75.2	99.8	96	58	99.9	99.75	72.7	93.27	100	90.42
DapQ 2026.03	100	100	96.4	99.8	93	58.4	99.9	99.85	71.82	93.6	100	92.07
H2O 2026.03	100	99.4	7.8	97	64.8	10.6	94.7	48.3	28.48	94.53	99.68	67.75
PyramidKV 2026.03	100	100	1.6	100	79.4	19	99.9	93.6	35.46	95.6	100	74.96
SnapKV 2026.03	100	100	20	100	91.6	30.4	99.85	97.75	49.59	96	100	80.47
DapQ 2026.03	100	100	55	100	88.05	31	99.95	95.15	45.76	96.07	100	82.82
PyramidKV 2026.03	100	99.8	0	98.6	61.8	2.4	97.75	70.2	17.84	92.47	99.12	67.27
SnapKV 2026.03	100	99.6	1	99.6	79.2	13	99.25	83.05	26.52	96.27	98.84	72.39
DapQ 2026.03	100	99.8	14.2	99.6	75.4	14.8	99.75	78.25	23.78	97.73	99.12	72.95
DapQ 2026.03	100	99.6	5.8	95.4	68.6	5.2	94.85	42.45	12.38	96.27	96.08	65.15
DapQ 2026.03	100	88.8	0	75.4	59.4	0.2	47.55	10.2	6.1	88	58.04	48.52
SnapKV 2026.03	99.6	99.2	0	97.2	60.6	2	94.35	47.25	14.78	93.6	98.64	64.29
PyramidKV 2026.03	99	97.2	0	84.4	40.8	0.2	70.7	30	6.2	70.27	70.92	51.79
H2O 2026.03	97.8	92	2.4	81.6	38.2	2.4	72	15.85	23.98	96.2	93.92	56.03
DapQ 2026.03	97.8	34	0	7	20.8	0	0.2	0.1	1.1	40.53	5.47	18.82
SnapKV 2026.03	97.4	87.8	0	81.6	35.2	0	47.2	13	9.7	84.47	72.32	48.06
PyramidKV 2026.03	94.8	60.4	0	41.6	14.4	0	5.8	5.6	1.22	34.07	24	25.63
H2O 2026.03	90.4	66.2	2.4	57.2	15	0.8	30.6	5.1	14.8	89.2	65.56	39.75
SnapKV 2026.03	68.8	31.2	0	2.6	3.2	0	0.15	0.45	2.06	40.33	5.24	14
H2O 2026.03	68	21.4	2.4	19	2.6	0	6.15	1.4	11.2	61.27	19.2	19.33
SLM 2026.03	26.6	28	27	24.4	19.6	18.4	26.15	27.1	36.62	93	22.96	31.8
H2O 2026.03	19.8	2.4	1.6	3.6	0.2	0	0.05	0.25	8.4	22.53	3.44	5.66
LaCache 2026.03	17.6	14.6	13.2	16.8	20.2	7.4	14.75	6.2	62.46	68.47	12.48	23.11
SLM 2026.03	13.8	13.4	11	11.4	9.4	9	10.85	11.3	10.38	95.47	9.72	18.7
PyramidKV 2026.03	11.4	2.8	0	0	1.2	0	0	0	0.84	4.6	3.68	2.23
SLM 2026.03	4.6	6.2	5.6	6.6	3.8	5	6.25	6.7	0.8	96.47	3.24	13.21
LaCache 2026.03	2	1.6	2.4	3.4	1.2	0	2.45	0.9	43.86	69.4	2.04	11.75
SLM 2026.03	1.8	4	1.2	3.8	2.4	3	3.75	4.4	0.76	97.6	0.56	11.21
SLM 2026.03	0.8	1.2	1.2	2	2.4	0	2.3	2.4	0.72	95	0	9.82
DapQ 2026.03	0.8	0.2	0	0	1.2	0	0	0	1	2.4	1.68	0.66
SLM 2026.03	0.6	1.2	1.2	2	0.2	0	2.3	2.4	0.64	96	0	9.69
LaCache 2026.03	0.2	1.6	2.4	3.2	0.4	0	2.45	1.05	15.86	62.07	0.32	8.14
LaCache 2026.03	0	1.6	0	3	0	0	0	0.8	7.06	26.07	0.08	3.51
LaCache 2026.03	0	0	0	0	0	0	0	0	1.7	7.93	0	0.88
LaCache 2026.03	0	0	0	0	0	0	0	0	0.46	0.2	0	0.06
LaCache 2026.03	0	0	0	0	0	0	0	0	0.72	0	0	0.07
SLM 2026.03	0	0	0	0	0	0	0	0	0.4	32.33	0	2.98
H2O 2026.03	0	0	0	0	0	0	0	0	3.42	0	1.24	0.42
PyramidKV 2026.03	0	0	0	0.2	0	0	0	0	0.78	0.4	0.56	0.18
SnapKV 2026.03	0	0	0	0	0	0	0	0	0.8	0.07	0.88	0.16