Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models

About

Block diffusion language models accelerate decoding through parallel block-wise denoising, yet whether they can be reliably scaled for long chain-of-thought (CoT) reasoning remains unresolved. To this end, we develop DreamReasoner-8B, an open-source block diffusion reasoning model, and conduct a systematic study of how training and inference block sizes affect long-CoT reasoning. Our analysis reveals a stark performance disparity: training with large block sizes yields remarkably poor reasoning, whereas small block sizes preserve effective reasoning. To bridge this granularity gap, we propose block-size curriculum learning, which gradually transitions training from fine-grained to coarse-grained block sizes, thereby overcoming this limitation and enabling strong reasoning performance that generalizes across diverse inference block sizes. On mathematical and code reasoning benchmarks, DreamReasoner-8B achieves results competitive with leading open autoregressive models such as Qwen3-8B. This work establishes a practical foundation for efficient, reasoning-capable diffusion language models. We release our model at https://github.com/DreamLM/DreamReasoner.

Zirui Wu, Lin Zheng, Jiacheng Ye, Shansan Gong, Xueliang Zhao, Yansong Feng, Wei Bi, Lingpeng Kong• 2026

Related benchmarks

TaskDatasetResultRank
Commonsense ReasoningHellaSwag
HellaSwag Accuracy76.6
897
Commonsense ReasoningPIQA
Accuracy81.2
400
Mathematical ReasoningGSM8K
Accuracy (Acc)83.4
352
Language UnderstandingMMLU
MMLU Accuracy75.4
307
Code GenerationHumanEval
Accuracy69.5
212
Reading ComprehensionRACE
Accuracy43.8
86
Question AnsweringARC-C
Accuracy (ARC-C)63.5
67
Commonsense ReasoningWinoGrande (WG)
Accuracy74.2
38
Code GenerationLiveCodeBench
Score53.9
29
Question AnsweringARC-E
Accuracy (ARC-E)87
11
Showing 10 of 10 rows

Other info

Follow for update