Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Mind-Paced Speaking: A Dual-Brain Approach to Real-Time Reasoning in Spoken Language Models

About

Real-time Spoken Language Models (SLMs) struggle to leverage Chain-of-Thought (CoT) reasoning due to the prohibitive latency of generating the entire thought process sequentially. Enabling SLMs to think while speaking, similar to humans, is attracting increasing attention. We present, for the first time, Mind-Paced Speaking (MPS), a brain-inspired framework that enables high-fidelity, real-time reasoning. Similar to how humans utilize distinct brain regions for thinking and responding, we propose a novel dual-brain approach, employing a "Formulation Brain" for high-level reasoning to pace and guide a separate "Articulation Brain" for fluent speech generation. This division of labor eliminates mode-switching, preserving the integrity of the reasoning process. Experiments show that MPS significantly outperforms existing think-while-speaking methods and achieves reasoning performance comparable to models that pre-compute the full CoT before speaking, while drastically reducing latency. Under a zero-latency configuration, the proposed method achieves an accuracy of 92.8% on the mathematical reasoning task Spoken-MQA and attains a score of 82.5 on the speech conversation task URO-Bench. MPS is the methodology underlying our released Step-Audio R1.1 system, effectively bridging the gap between high-quality reasoning and real-time interaction.

Donghang Wu, Haoyang Zhang, Jun Chen, Xiangyu (Tony) Zhang, Hexin Liu, Eng Siong Chng, Fei Tian, Xuerui Yang, Xiangyu Zhang, Daxin Jiang, Gang Yu• 2025

Related benchmarks

TaskDatasetResultRank
Spoken Multi-choice Question AnsweringSpoken-MQA (test)
Arithmetic Average Accuracy89
15
Speech ConversationURO-Bench Chinese Basic
Understanding Score93.6
9
Speech ConversationURO-Bench Chinese Pro
Understanding (U)77.2
9
Speech ConversationURO-Bench English Basic
Understanding Score (U)94.2
9
Speech ConversationURO-Bench English Pro
Understanding Score76.5
9
Showing 5 of 5 rows

Other info

GitHub

Follow for update