Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models

About

Large reasoning models rely on long chain-of-thought to achieve strong performance, but applying such reasoning uniformly incurs high computational cost. Existing efficiency-oriented methods attempt to shorten or mix reasoning strategies, yet often degrade reasoning capability. We identify the root cause as sequence-level coupling between efficiency incentives and correctness optimization, which implicitly penalizes long but correct reasoning trajectories. To address this issue, we propose Adaptive Dual-Process Thinking (ADaPT), a token-level dual-process framework that explicitly decouples efficiency and correctness signals during training. ADaPT introduces a mode-selection token to control fast and slow reasoning, applying efficiency-related rewards exclusively to this token to avoid penalizing correct long reasoning while encouraging efficiency when appropriate. Moreover, ADaPT enables precise and continuous control over the efficiency-performance trade-off at inference time: by adjusting the generation probability of the mode-selection token, a single trained model can smoothly move along the efficiency-performance Pareto frontier. Extensive experiments demonstrate that ADaPT significantly reduces inference cost while maintaining strong reasoning performance across multiple benchmarks.

Tingyun Li, Zishang Jiang, Jinyi Han, Xinyi Wang, Sihang Jiang, Han Xia, Zhaoqian Dai, Shuguang Ma, Fei Yu, Jiaqing Liang, Yanghua Xiao• 2026

Related benchmarks

TaskDatasetResultRank
ReasoningARC
Accuracy87.7
269
Mathematical ReasoningMATH500
Accuracy78.2
104
Mathematical ReasoningOlympiad
Pass@1 Accuracy40.2
60
Question AnsweringCSQA
Accuracy (CSQA)86.7
24
General ReasoningAggregate Easy and Hard Benchmarks
Accuracy63.7
18
Massive Multitask Language UnderstandingMMLU-Pro
Accuracy49.8
18
Math Word ProblemsGSM8K
Accuracy91.4
18
Mathematical ReasoningAIME 2024
Accuracy (ACC)15.1
18
Showing 8 of 8 rows

Other info

Follow for update