Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Reasoning on LongBench V2 (DocMath, Frames, LB-MQA Test)

66.7DocMath Score

Qwen3-30B-A3B-Thinking + SPELL

Updated 1mo ago

Evaluation Results

Method
Qwen3-30B-A3B-Thinking + SPELL 2025.09	66.7	68.1	78.4	50.5	65.9
Qwen3-30B-A3B-Instruct + SPELL 2025.09	64.9	63.7	74.8	48.7	63
Qwen3-4B-Thinking + SPELL 2025.09	64.8	60.6	72.4	43	60.2
Qwen3-30B-A3B-Instruct + RLVR 2025.09	64	62	72.4	47.4	61.5
Qwen3-30B-A3B-Thinking + RLVR 2025.09	63.9	67.1	77.2	49.6	64.5
Qwen3-30B-A3B-Thinking 2025.09	63.8	65.8	77.9	46.7	63.6
Qwen2.5-32B-Instruct + SPELL 2025.09	63.3	62	74.1	40.8	60.1
Qwen3-4B-Thinking + RLVR 2025.09	63.3	58.6	71.1	43.4	59.1
Qwen2.5-32B-Instruct 2025.09	63	49.4	61.5	36.2	52.5
Qwen3-30B-A3B-Instruct 2025.09	63	57.8	70.3	44.1	58.8
Qwen3-4B-Thinking 2025.09	61.4	59.2	70.9	40.7	58.1
R1-Distill-Qwen-14B + SPELL 2025.09	61.1	62.8	75.7	37.9	59.4
R1-Distill-Qwen-14B + RLVR 2025.09	61	63.8	76	35.9	59.2
Qwen2.5-32B + SPELL 2025.09	60.6	52.2	62.3	34.3	52.4
Qwen2.5-14B-Instruct + SPELL 2025.09	60.1	63.9	74.8	40.1	59.7
Qwen2.5-32B-Instruct + RLVR 2025.09	59.7	62.3	69.6	36.9	57.1
R1-Distill-Qwen-14B 2025.09	59.5	60.6	73.3	33.3	56.7
Qwen2.5-32B + RLVR 2025.09	57.5	49.9	60.1	32.7	50.1
Qwen2.5-14B + SPELL 2025.09	56.8	53	63.2	31.2	51.1
Qwen2.5-14B-Instruct 2025.09	56.7	52.4	64.2	36.6	52.5
Qwen2.5-14B-Instruct + RLVR 2025.09	56.7	59.9	73.4	38.5	57.1
Qwen2.5-14B + RLVR 2025.09	53.2	52.1	64.2	30.5	50
R1-Distill-Llama-8B + SPELL 2025.09	49.2	54.3	70	29.3	50.7
R1-Distill-Llama-8B + RLVR 2025.09	45.4	54	68	28.3	48.9
Qwen2.5-7B-Instruct + SPELL 2025.09	44.5	48.2	60.7	32.4	46.5
Qwen2.5-7B-Instruct + RLVR 2025.09	44.1	48.6	57.4	28.2	44.6
Qwen2.5-7B + RLVR 2025.09	42.7	40.3	49.2	26	39.6
R1-Distill-Llama-8B 2025.09	41.5	52.6	69.3	26.4	47.5
Qwen2.5-32B 2025.09	40.7	42.2	50.1	28.7	40.4
Qwen2.5-7B + SPELL 2025.09	39.9	40.1	50.8	28.2	39.8
Llama3.1-8B-Instruct + SPELL 2025.09	39.7	50.8	60.9	26.2	44.4
Qwen2.5-7B-Instruct 2025.09	39.4	41.4	44.5	28.4	38.4
Llama3.1-8B-Instruct + RLVR 2025.09	36.9	47.6	57.2	26.1	42
Qwen2.5-14B 2025.09	36.2	37.5	43.3	27.5	36.1
Llama3.1-8B-Instruct 2025.09	34.9	47.3	53.5	27.1	40.7
Qwen2.5-7B 2025.09	16.1	24.2	31.2	22.7	23.6