Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesis

About

Building state-of-the-art text-to-speech (TTS) systems typically demands millions of hours of proprietary data and complex multi-stage architectures, creating substantial barriers for resource-constrained research teams. In this report, we present PilotTTS, a lightweight autoregressive TTS system that achieves competitive performance through minimalist architecture and rigorous data engineering. PilotTTS is trained on only 200K hours of data processed entirely with open-source tools. Specifically, our contributions are: (1) a reproducible multi-stage data processing pipeline covering quality assessment, label annotation, and filtering, and (2) a compact model architecture that employs Q-Former-based conditioning to decouple speaker identity from speaking style via cross-sample paired training. Within a unified framework, PilotTTS supports zero-shot voice cloning, emotion synthesis (11 categories), paralinguistic synthesis (4 categories), and Chinese dialect synthesis (14 dialects). On the Seed-TTS Eval benchmark, PilotTTS achieves the lowest WER of 1.50% on test-en, a CER of 0.87% on test-zh, and the highest speaker similarity on both test sets (0.862 and 0.815), outperforming systems trained on significantly larger datasets. We release the complete data pipeline recipe, pretrained weights, and code at https://github.com/AMAPVOICE/PilotTTS.

Bowen Li, Shaotong Guo, Zhen Wang, Yang Xiang, Mingli Jin, Yihang Lin, Jiahui Zhao, Weibo Xiong, Dongrui Zhang, Keming Chen, Yunze Gao, Zeyang Lin, Yuze Zhou, Yue Liu• 2026

Related benchmarks

TaskDatasetResultRank
Speaker Similarity51 speaker prompts Emotion Control evaluation
Speaker Similarity0.8101
10
Zero-shot Speech GenerationSeed-TTS-Eval en (test)
WER (%)1.5
9
Zero-shot Speech GenerationSeed-TTS-Eval zh (test)
CER0.87
9
Emotion ControlEmotion Control Evaluation Set 51 speaker prompts
Happy86.4
5
Paralinguistic synthesisDedicated paralinguistic (test)
Laughter Score97.6
3
Showing 5 of 5 rows

Other info

Follow for update