Our new X account is live! Follow @wizwand_team for updates

Long-context understanding on LongBench (Task-Specific Metrics)

62.1Overall Average Score

DPO w/ LongReward

Updated 4d ago

Evaluation Results

Method
DPO w/ LongReward 2024.10	62.1	71.9	58.8	48.5	-	-	-	-	-
DPO w/ Contrast 2024.10	60.5	67.8	58	47.8	-	-	-	-	-
officially post-trained 2024.10	60.3	67.8	56.9	47.9	-	-	-	-	-
DPO w/ LongReward 2024.10	59.9	67.8	55.8	43.2	-	-	-	-	-
DPO w/ SRM 2024.10	59.9	67.5	57.4	48.2	-	-	-	-	-
SFT 2024.10	56.6	68.4	50.9	42.1	-	-	-	-	-
DPO w/ SRM 2024.10	56.2	65	49.6	42.7	-	-	-	-	-
DPO w/ Contrast 2024.10	56.2	67.8	46.2	40.3	-	-	-	-	-
SFT 2024.10	55	66.1	44.5	39.6	-	-	-	-	-
DeltaKV† 2026.02	50.3	43.2	46.8	27.7	69.5	54.8	59.7	45	30
DeltaKV 2026.02	50.3	43.3	46.6	27.3	69.8	54.4	60.5	29	30
OmniKV 2026.02	50.2	44.8	46.1	28.9	68.9	52.8	59.9	100	30
DeltaKV 2026.02	50.2	44.4	45.9	27.6	69.7	53.4	60.2	45	30
OmniKV 2026.02	50.1	43	45.7	27.5	68.6	54.9	60.8	100	20
GPT-4 2024.08	50	-	-	-	-	-	-	-	-
Llama-3.1-8B 2026.02	50	45.3	46.2	28.7	69.4	52.7	57.9	100	100
Quest 2026.02	50	44.4	46.2	29.3	69	53.1	57.6	100	30
SnapKV 2026.02	49.8	44.5	46.4	26.5	68.2	52.7	60.3	30	30
DeltaKV† 2026.02	49.8	42.7	46.2	26.2	68.7	54.3	60.5	43	20
AdaKV 2026.02	49.7	44.7	46.5	26.6	69.3	52.7	58.3	30	30
Quest 2026.02	49.7	42.2	46.9	28.8	68.9	54.9	56.5	100	20
PyramidKV 2026.02	49.5	44.1	46.3	26	68.5	52.6	59.1	30	30
officially post-trained 2024.10	49.4	59.3	42.9	35.3	-	-	-	-	-
SnapKV 2026.02	49.2	42.9	45.4	25.8	68.8	54.8	57.4	20	20
Gemini-1.5-Pro 2024.08	48.5	-	-	-	-	-	-	-	-
ChatGLM3-6B 2024.03	48.4	40.9	45	26	56.5	65	57.1	-	-
Qwen2.5-32B 2026.02	48.4	42.7	54.3	27.3	67.6	56	42.6	100	100
InternLM2-Chat-7B-SFT 2024.03	48.1	47.3	45.2	25.3	59.9	67.2	43.5	-	-
OmniKV 2026.02	48.1	42.4	54.2	26.9	67.2	56.1	41.7	100	20
DeltaKV 2026.02	47.7	41.9	54.2	26.2	66.1	55.4	42.1	44	20
Qwen2.5-7B 2026.02	47.6	42.5	49.6	28.6	68.7	53.8	42.5	100	100
OmniKV 2026.02	47.4	41.9	49.4	28.4	69.1	54	41.5	100	30
DeltaKV 2026.02	47.1	41.8	49	27.7	69.1	53.3	41.7	48	30
SnapKV 2026.02	47.1	39.5	53.8	24.6	67.2	56	41.7	20	20
SnapKV 2026.02	47	41.7	48.8	26.6	68.3	54.5	41.8	30	30
InternLM2-Chat-20B-SFT 2024.03	46.8	46.8	48.7	25.6	51.2	67.9	40.4	-	-
InternLM2-Chat-7B 2024.03	46.1	45.7	43.1	26.5	58.3	66.3	36.4	-	-
PyramidKV 2026.02	46.1	40.6	48.7	24.4	67.7	54.5	40.6	30	30
InternLM2-Chat-20B 2024.03	44.8	46.9	46.7	26	49.1	67.3	32.6	-	-
Original Prompt 2025.07	43.03	37.35	36.36	27.39	71.01	-	-	-	-
Llama3.1-70B-Instruct 2024.08	42.8	-	-	-	-	-	-	-	-
Full 2026.02	41.47	47.51	43.28	25.9	45.92	68.18	18.01	-	-
MInference 2026.02	41.14	47.42	42.54	25.85	45.58	67.6	17.84	-	-
Prism 2026.02	41.08	47.09	42.13	26	46.4	66.15	18.72	-	-
XAttention 2026.02	39.68	45.89	41.56	26.18	45.86	59.32	19.24	-	-
Full 2026.02	39.49	47.1	40.45	24.07	56.69	67	1.65	-	-
MInference 2026.02	39.18	46.9	40.39	24.07	55.74	66.33	1.61	-	-
Prism 2026.02	39.12	46.47	40.08	24.01	58.36	64.17	1.64	-	-
XAttention 2026.02	38.82	44.49	40.09	24.12	57.27	65.67	1.29	-	-
Palu 2026.02	38.8	34.4	35.6	27.5	68.7	45	21.3	50	100
Mistral-7B-Instruct-v0.2 2024.03	38.3	31.3	26.4	21.8	46.6	59.2	44.8	-	-
Mixtral-8x7B-Instruct-v0.1 2024.03	37	35	25.6	17.1	35.9	68.2	40	-	-
FlexPrefill 2026.02	36.13	43.77	39.31	23.99	57.33	50.5	1.87	-	-
CREAM-7B-32k 2024.06	35.9	34.8	31.1	27.2	65.1	7	50.4	-	-
DAC 2025.07	35.42	32.68	28.08	22.92	58.01	-	-	-	-
LLMLingua-2 2025.07	34.82	32.51	27.93	22.74	56.08	-	-	-	-
LongChat-v1.5-7B-32k 2024.06	34.3	28.7	20.6	26.7	60	15.8	54.1	-	-
FlexPrefill 2026.02	33.9	46.13	41.49	25.85	46.63	25.61	17.68	-	-
LLMLingua 2025.07	33.31	27.81	24.68	23.13	57.63	-	-	-	-
Selective-Context 2025.07	32.16	26.4	24.78	21.98	55.47	-	-	-	-
Qwen2-7B-Instruct 2024.08	31.9	-	-	-	-	-	-	-	-
Vicuna-v1.5-7B-16k 2024.06	31.9	28	18.6	26	66.2	5.5	47.3	-	-
Gradient-Llama3-70B 2024.08	31.8	-	-	-	-	-	-	-	-
WINA 2025.05	31.39	21.19	15.98	13.39	59.39	48.76	36.53	-	-
Qwen2-72B-Instruct 2024.08	31	-	-	-	-	-	-	-	-
Llama2-7B-chat-4k 2024.06	31	24.9	22.6	24.7	60	5.9	48.1	-	-
Baichuan2-13B-Chat 2024.03	30.9	34.3	29.1	21.3	45.5	4	51.4	-	-
TEAL 2025.05	30.54	22.15	11.69	15.04	58.44	51.03	29.51	-	-
Yi-9B-200k 2024.08	30.3	-	-	-	-	-	-	-	-
WINA 2025.05	30.26	16.3	12.24	17.36	59.22	46.58	37.6	-	-
TEAL 2025.05	30.07	19.25	14.11	19.5	59.48	43.52	25.71	-	-
Llama3.1-8B-Instruct 2024.08	30	-	-	-	-	-	-	-	-
Qwen-14B-Chat 2024.03	29.7	32.6	19.4	22.5	36.9	23.7	42.9	-	-
WINA 2025.05	29.43	19.6	11.28	11.68	56.13	52.1	33.51	-	-
GLM4-9B-Chat-1M 2024.08	29.4	-	-	-	-	-	-	-	-
Yi-34B-200k 2024.08	29.1	-	-	-	-	-	-	-	-
Mistral-7B-Instruct-v0.2 2024.06	29	28.5	21.5	26.1	50.1	13.9	33.8	-	-
Mistral-7B-Instruct-v0.3 2024.06	28.8	33.2	30.6	26.8	56.4	10.4	15.3	-	-
TEAL 2025.05	28.74	21.38	9.95	12.53	55.2	48.7	30.65	-	-
XGen-7B-8k 2024.06	28.3	24.6	20.4	24.7	56.2	5.3	38.6	-	-
WINA 2025.05	28.3	19.34	6.48	9.07	55.53	57.46	30.15	-	-
FLT* 2024.08	28.2	-	-	-	-	-	-	-	-
POSE 2024.08	28.2	-	-	-	-	-	-	-	-
FLT* 2024.08	28.2	-	-	-	-	-	-	-	-
FLT 2024.08	28.1	-	-	-	-	-	-	-	-
Benchmark 2025.05	28.06	18.59	4.8	8.41	56.03	59.59	29.58	-	-
RPES 2024.08	27.9	-	-	-	-	-	-	-	-
TEAL 2025.05	27.86	18.32	5.73	10.11	54.83	55.6	31.16	-	-
RPES 2024.08	27.8	-	-	-	-	-	-	-	-
Gradient-Llama3-8B 2024.08	27.8	-	-	-	-	-	-	-	-
POSE 2024.08	27.7	-	-	-	-	-	-	-	-
RPES 2024.08	27.3	-	-	-	-	-	-	-	-
POSE 2024.08	27.2	-	-	-	-	-	-	-	-
LongRecipe 2024.08	26.9	-	-	-	-	-	-	-	-
RPES 2024.08	26.8	-	-	-	-	-	-	-	-
POSE 2024.08	26.7	-	-	-	-	-	-	-	-
FLT 2024.08	26.4	-	-	-	-	-	-	-	-
LongRecipe 2024.08	26.2	-	-	-	-	-	-	-	-
Baichuan2-7B-Chat 2024.03	25.9	30.3	18.4	21.8	42.5	6.6	36	-	-
LongRecipe 2024.08	25.7	-	-	-	-	-	-	-	-

Showing 100 of 115 rows