Share your thoughts, 1 month free Claude Pro on usSee more

Text-to-Speech on EmergentTTS (eval)

6.35Overall WER

Gemini-2.5-Flash-Preview-TTS

Updated 1mo ago

Evaluation Results

Method
Gemini-2.5-Flash-Preview-TTS 2026.03	6.35	79.1	0.71	97.32	11.8	65.56	18.38	91.25	0.4	74.82	0.5	67.14	-
Gemini-2.5-Flash-Preview-TTS 2026.03	6.59	78	0.6	95	12.99	63.03	18.25	89.1	0.36	74.82	0.73	68.03	-
minimax/speech-02-hd 2026.03	6.79	40.96	0.57	41.6	14.58	31.96	17.69	33.57	0.27	52.14	0.84	45.53	-
gpt-4o-mini-tts 2026.03	7.11	61.98	0.71	67.32	12.07	63.03	21.33	63.39	0.66	54.28	0.84	61.96	-
gpt-4o-mini-audio-preview 2026.03	7.18	55.15	0.95	62.9	14.48	64.82	19.04	52.14	0.55	45.71	0.88	50.17	-
Baseline gpt-4o-mini-tts 2026.03	7.23	50	0.72	-	13.45	-	20.55	-	0.42	-	1.04	-	-
Fish Audio S1 2026.03	7.6	36.88	0.75	24.82	15.77	53.75	20.02	37.68	0.72	43.55	0.74	24.64	-
gpt-4o-audio-preview 2026.03	7.64	64.9	0.93	67.14	13.75	71.6	20.56	77.32	1.72	49.64	1.26	58.92	-
11Labs eleven multilingual v2 2026.03	7.64	36.96	0.63	35.35	14.44	36.6	21.51	52.14	0.49	28.21	1.15	32.5	-
gpt-4o-audio-preview 2026.03	7.75	77.36	1.82	96.96	13.3	70	21.15	89.46	1.38	62.5	1.16	68.39	-
Gemini-2.5-Pro-Preview-TTS 2026.03	7.91	73	0.87	92.14	16.22	63.44	20.87	84.28	0.72	65.71	0.87	60	-
Fish Audio S2 2026.03	8.15	81.88	0.52	86.61	14.74	63.39	24.58	91.61	0.38	84.41	0.51	83.39	-
ResembleAI Chatterbox 2026.03	8.2	27.39	1.18	28.03	17.59	24.64	20.64	17.32	0.65	51.07	0.96	15.89	-
gpt-4o-audio-preview 2026.03	8.27	60.53	1.03	56.78	14.72	68.75	23.16	71.6	1.19	48.75	1.25	56.78	-
HumeAI 2026.03	8.62	49.12	0.83	71.78	21.05	41.25	19.84	42.14	0.38	42.14	0.93	48.57	-
gpt-4o-mini-audio-preview 2026.03	8.86	66.27	9.34	74.1	12.7	73.39	20.92	68.03	0.74	55.53	0.72	60.35	-
KyutAI-TTS 2026.03	9.1	24.28	0.83	31.78	16.47	13.39	26.41	24.28	0.61	35.17	1.19	16.78	-
MiniMax/speech-02-hd⋆ 2026.06	10.02	36.6	-	40.9	-	34.3	-	34.3	-	47.3	-	43.9	16.3
DeepGram Aura-2 2026.03	10.24	25.21	3.45	17.5	21.41	15.89	23.73	20.89	1.24	43.03	1.36	28.75	-
Gemini-2.5-Flash-TTS⋆ 2026.06	10.39	70.7	-	95.9	-	58.5	-	91.3	-	63	-	57.9	55.7
dots.tts (SOAR) 2026.06	10.45	47.6	-	63.9	-	39.4	-	52.7	-	47	-	65.7	16.4
Kokoro-82M 2026.03	10.49	25.46	0.71	18.03	22.17	13.21	28.37	5.89	0.56	43.39	0.65	46.78	-
BASELINE: gpt-4o-mini-tts 2026.06	10.61	50	-	-	-	-	-	-	-	-	-	-	-
gpt-4o-mini-tts⋆ 2026.06	10.76	56.3	-	59.2	-	57.3	-	58.8	-	52.7	-	57.1	52.4
dots.tts (Pretrain) 2026.06	10.86	49.2	-	72.7	-	39.5	-	54.7	-	48.4	-	58.4	18
11Labs Multilingual v2⋆ 2026.06	11.19	33.9	-	30.4	-	35.5	-	45.5	-	39.5	-	35.5	14.5
dots.tts (MF4) 2026.06	11.75	47.9	-	59.8	-	36.3	-	55.2	-	50.5	-	64.8	16.7
Gemini-2.5-Pro-TTS⋆ 2026.06	11.79	69.3	-	86.9	-	58.2	-	82.3	-	61.3	-	61.8	64.8
Higgs Audio V2 2026.03	11.82	44.64	0.97	69.28	22.26	21.42	31.34	45.53	3.23	45.35	1.29	41.6	-
VoxCPM2 2026.06	11.84	41.1	-	42.3	-	33.3	-	44.1	-	53.4	-	52.3	18.6
gpt-4o-audio-preview⋆ 2026.06	11.87	65.2	-	88.8	-	60.2	-	82.1	-	57	-	59.5	40.4
F5TTS 2026.03	11.93	19.43	0.7	29.64	23.51	3.21	31.66	19.82	1.62	19.46	2.14	25	-
HumeAI⋆ 2026.06	12.85	42.7	-	61.6	-	34.6	-	36.9	-	43.2	-	44.6	34.3
Orpheus TTS 2026.03	13.63	32.56	1.81	39.06	22.31	14.64	40.94	48.57	1.48	31.07	1.63	29.46	-
F5-TTS 2026.06	16.47	15.3	-	26.8	-	1.8	-	21.6	-	14.8	-	23.8	1.4
Qwen3-TTS 2026.06	17.32	42.8	-	39.8	-	25.4	-	50.7	-	48.9	-	60.4	30
Qwen 2.5 Omni 2026.03	18.34	32.94	2.41	46.6	26.77	14.46	58.44	21.25	0.87	48.92	3.47	33.75	-
Qwen3-TTS 2026.06	19.65	42.3	-	60.5	-	17.1	-	62.7	-	56.4	-	43	9.8
Qwen 2.5 Omni 2026.03	20.02	30.25	1.22	41.07	26.98	12.5	57.48	20.89	12.77	49.1	1.66	27.67	-
MiniCPM-o 2026.03	22.52	19.75	12.36	22.85	33.46	5.89	58.48	21.6	5.21	22.14	3.08	26.25	-