Share your thoughts, 1 month free Claude Pro on usSee more

Long-context Reasoning on Long-context Benchmarks 16K (DocMath, Frames, LB-MQA, V2) (test)

64.1DocMath

Qwen3-30B-A3B-Thinking + SPELL

Updated 1mo ago

Evaluation Results

Method
Qwen3-30B-A3B-Thinking + SPELL 2025.09	64.1	66.5	78	42.3	62.7
Qwen3-30B-A3B-Instruct + SPELL 2025.09	63	63.1	75.1	41.5	60.7
Qwen3-30B-A3B-Thinking 2025.09	62.9	64.5	75.7	39.7	60.7
Qwen3-30B-A3B-Thinking + RLVR 2025.09	62.7	64.7	77	38.5	60.7
Qwen3-30B-A3B-Instruct + RLVR 2025.09	62.5	59.9	71.8	39.8	58.5
Qwen2.5-32B-Instruct + SPELL 2025.09	62.3	61.2	74.4	40.1	59.5
Qwen3-30B-A3B-Instruct 2025.09	62.3	55.3	70.5	36.9	56.3
Qwen3-4B-Thinking + SPELL 2025.09	61.9	56.6	71.6	36.8	56.7
Qwen2.5-32B + SPELL 2025.09	61.8	50.2	62.1	34.2	52.1
R1-Distill-Qwen-14B + SPELL 2025.09	61.6	62.3	76.2	39	59.8
Qwen3-4B-Thinking + RLVR 2025.09	60.5	56.6	71.1	33.8	55.5
Qwen2.5-32B-Instruct 2025.09	60	49.9	61.4	36	51.8
Qwen2.5-32B-Instruct + RLVR 2025.09	59.9	60.5	70.4	36.3	56.8
Qwen2.5-14B-Instruct + SPELL 2025.09	59.6	62.1	72.8	36.8	57.8
R1-Distill-Qwen-14B + RLVR 2025.09	59.6	61.7	74.6	37.2	58.3
Qwen3-4B-Thinking 2025.09	58.6	56.7	69.9	32.9	54.5
Qwen2.5-32B + RLVR 2025.09	58.3	50	59.5	32.8	50.2
R1-Distill-Qwen-14B 2025.09	57.7	59.2	72.4	36.2	56.4
Qwen2.5-14B + SPELL 2025.09	57.6	52.6	63	33.5	51.7
Qwen2.5-14B-Instruct 2025.09	56.3	51.6	63	32.2	50.8
Qwen2.5-14B-Instruct + RLVR 2025.09	56.1	59.6	71	36.4	55.8
Qwen2.5-14B + RLVR 2025.09	52.2	51	63.3	32.9	49.9
R1-Distill-Llama-8B + SPELL 2025.09	48.9	53.4	68.4	30.2	50.2
Qwen2.5-32B 2025.09	46.8	42.6	49	33.7	43
Qwen2.5-7B-Instruct + SPELL 2025.09	45.8	46.7	63.1	33.2	47.2
Qwen2.5-7B-Instruct + RLVR 2025.09	45	48.7	59.6	30.1	45.9
R1-Distill-Llama-8B + RLVR 2025.09	43.4	51.4	67.8	30	48.2
R1-Distill-Llama-8B 2025.09	42	50.3	66.8	27.9	46.8
Qwen2.5-7B + RLVR 2025.09	41.8	41	50	30.2	40.8
Qwen2.5-7B + SPELL 2025.09	40	39.2	50.9	32.3	40.6
Llama3.1-8B-Instruct + SPELL 2025.09	39.2	48.9	61.6	28.4	44.5
Qwen2.5-7B-Instruct 2025.09	38.4	40.3	45.1	29	38.2
Qwen2.5-14B 2025.09	38	37.2	41.9	32.1	37.3
Llama3.1-8B-Instruct + RLVR 2025.09	37.9	45	58.8	27.5	42.3
Llama3.1-8B-Instruct 2025.09	33.2	45.6	52.5	29.1	40.1
Qwen2.5-7B 2025.09	10.9	27.9	36.7	31.2	26.7