Our new X account is live! Follow @wizwand_team for updates

Long-context evaluation on RULER 64k

100VT Score

Llama-3.1-8B

Updated 4d ago

Evaluation Results

Method
Llama-3.1-8B 2025.05	100	100	100	100	100	100	96	99	100	14.8	92	60	52	85.6	-	-
Qwen3-4B-SFT w/ LongR 2026.02	93.32	-	-	-	-	-	-	-	-	-	-	-	-	69.43	71.36	43.6
Qwen3-4B-SFT w/ DAPO 2026.02	91.96	-	-	-	-	-	-	-	-	-	-	-	-	63.53	67.74	30.9
Qwen3-4B-SFT 2026.02	90.92	-	-	-	-	-	-	-	-	-	-	-	-	59.11	66.2	20.2
TailorKV-2 2025.05	88	100	100	100	100	96	68	97	98	19.6	62.7	60	56	80.4	-	-
Qwen3-8B-SFT w/ LongR 2026.02	86.64	-	-	-	-	-	-	-	-	-	-	-	-	71.98	69.88	59.41
TailorKV-1 2025.05	85.6	100	100	96	100	96	28	99	100	18.4	57.3	56	48	75.7	-	-
Quest 2025.05	80	88	100	60	92	72	0	93	90	8.4	70.6	52	52	66	-	-
SnapKV 2025.05	74.4	96	84	0	88	32	0	40	69	0.8	41.3	56	48	48.4	-	-
Qwen3-8B-SFT w/ DAPO 2026.02	58.52	-	-	-	-	-	-	-	-	-	-	-	-	55.02	66.13	40.4
Qwen3-4B-Base 2026.02	56.4	-	-	-	-	-	-	-	-	-	-	-	-	45.42	60.55	19.3
PQCache 2025.05	52.8	36	60	12	68	48	4	16	37	0	73.3	56	48	39.2	-	-
TailorKV-2 2025.05	42.4	100	100	92	100	84	28	62	90	33.6	48	28	48	65.8	-	-
TailorKV-2 2025.05	42.4	100	100	100	100	24	28	40	67	0.8	57.3	32	24	55.1	-	-
PQCache 2025.05	31.2	32	56	4	36	16	0	7	39	6	66.6	20	36	26.9	-	-
Qwen3-8B-SFT 2026.02	31	-	-	-	-	-	-	-	-	-	-	-	-	44.25	62.96	38.8
TailorKV-1 2025.05	28.8	100	100	100	96	12	24	41	65	1.2	58.7	28	24	52.2	-	-
Qwen3-8B-Base 2026.02	28.08	-	-	-	-	-	-	-	-	-	-	-	-	43.18	61.76	39.7
Yi-6B 2025.05	24.8	100	100	100	96	56	24	39	76	0.8	73.3	32	24	57.3	-	-
SnapKV 2025.05	22.4	80	28	0	20	4	0	11	11	5.6	48	24	44	22.9	-	-
Quest 2025.05	20	72	84	0	52	20	0	28	30	1.6	56	24	20	31.3	-	-
Quest 2025.05	16	68	92	20	68	40	0	24	42	10.8	62.6	28	36	39	-	-
SnapKV 2025.05	15.2	88	4	0	16	0	0	5	7	0	65.3	28	20	19.1	-	-
Yi-9B 2025.05	12.8	100	100	100	100	92	48	61	88	15.6	88	32	48	68.1	-	-
PQCache 2025.05	10.4	16	20	0	28	8	0	5	3	0	50.6	24	24	14.5	-	-
TailorKV-1 2025.05	6.4	100	100	92	100	84	28	53	89	32	49.3	32	48	62.6	-	-
StreamLLM 2025.05	2.4	8	4	0	8	0	0	5	3	0.8	72	28	28	12.2	-	-
StreamLLM 2025.05	0	0	4	0	4	0	0	1	0	1.2	74.6	16	28	9.9	-	-
StreamLLM 2025.05	0	0	0	0	8	0	0	3	0	0.4	62.6	20	16	8.5	-	-