Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

TiCo: Time-Controllable Spoken Dialogue Model

About

We introduce TiCo, a time-controllable spoken dialogue model (SDM) that follows time-constrained instructions (e.g., "Please generate a response lasting about 15 seconds") and generates spoken responses with controllable duration. This capability is valuable for real-world spoken language systems such as voice assistants and interactive agents, where controlling response duration can improve interaction quality. However, despite their strong ability to generate natural spoken responses, existing models lack time awareness and struggle to follow duration-related instructions. To systematically evaluate this, we introduce TiCo-Bench, the first benchmark for time-controllable instruction following in SDMs, on which existing open-source and commercial models frequently fail to satisfy explicit time constraints. TiCo addresses this limitation by enabling an SDM to estimate elapsed speaking time during generation through Spoken Time Markers (STM) (e.g., <10.6 seconds>). These markers help the model maintain awareness of time and adjust the remaining content to meet the target duration. TiCo is post-trained efficiently without question-answer paired data, relying on self-generation and reinforcement learning with verifiable reward. Experimental results show that TiCo reduces duration error by 2.7x over its backbone and 1.6x over the strongest baseline, while preserving response quality.

Kai-Wei Chang, Wei-Chih Chen, En-Pei Hu, Hung-yi Lee, James Glass• 2026

Related benchmarks

TaskDatasetResultRank
Time-controlled instruction-following (text-query)TiCo-Bench Text Short (10s–30s)
MAE (s)4.08
14
Time-controlled instruction-following (text-query)TiCo-Bench Text Long (30s–1min)
MAE (s)6.25
14
Time-controlled instruction-followingInstructS2S short (10s–30s)
MAE3.16
8
Time-controlled instruction-followingUROBench short (10s–30s)
MAE3.71
8
Time-controlled instruction-followingInstructS2S long (30s–1min)
MAE5.16
8
Time-controlled instruction-followingUROBench long (30s–1min)
MAE6.13
8
Time-controlled instruction-followingTiCo-Bench Speech Overall
MAE4.54
8
Time-controlled instruction-followingTiCo-Bench Speech
InstructS2S (Short 10s-30s)3.39
8
Time-controlled instruction-followingTiCo-Bench Text Overall
MAE (seconds)5.35
7
Time-controlled instruction-following (text-query)TiCo-Bench Text
Overall Time (s)2.76
7
Showing 10 of 10 rows

Other info

Follow for update