Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models

About

Recent advances in duplex speech models have enabled natural, low-latency speech-to-speech interactions. However, existing models are restricted to a fixed role and voice, limiting their ability to support structured, role-driven real-world applications and personalized interactions. In this work, we introduce PersonaPlex, a duplex conversational speech model that incorporates hybrid system prompts, combining role conditioning with text prompts and voice cloning with speech samples. PersonaPlex is trained on a large-scale synthetic dataset of paired prompts and user-agent conversations, generated with open-source large language models (LLM) and text-to-speech (TTS) models. To evaluate role conditioning in real-world settings, we extend the Full-Duplex-Bench benchmark beyond a single assistant role to multi-role customer service scenarios. Experiments show that PersonaPlex achieves strong role-conditioned behavior, voice-conditioned speech, and natural conversational responsiveness, surpassing state-of-the-art duplex speech models and hybrid large language model-based speech systems in role adherence, speaker similarity, latency, and naturalness.

Rajarshi Roy, Jonathan Raiman, Sang-gil Lee, Teodor-Dumitru Ene, Robert Kirby, Sungwon Kim, Jaehyeon Kim, Bryan Catanzaro• 2026

Related benchmarks

TaskDatasetResultRank
Interruption HandlingFull-Duplex-Bench
GPT-4o Score4.21
18
Turn TakingFull-Duplex-Bench
TOR99.2
17
Pause HandlingFull-Duplex-Bench Candor
TOR0.662
13
User InterruptionBilingual Full-Duplex-Bench English
RL0.4
12
Pause HandlingFull-Duplex-Bench Synthetic
TOR58.4
11
BackchannelingFull-Duplex-Bench
TOR32.7
11
Overall EvaluationBilingual Full-Duplex-Bench English
Accuracy79
8
Duplex Dialogue Turn-TakingFull-Duplex-Bench
Synthetic TOR for Pause Handling0.358
8
Turn TakingBilingual Full-Duplex-Bench English
TOR99.2
6
Pause HandlingBilingual Full-Duplex-Bench English
TOR62.3
6
Showing 10 of 15 rows

Other info

Follow for update