Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

BigVGAN: A Universal Neural Vocoder with Large-Scale Training

About

Despite recent progress in generative adversarial network (GAN)-based vocoders, where the model generates raw waveform conditioned on acoustic features, it is challenging to synthesize high-fidelity audio for numerous speakers across various recording environments. In this work, we present BigVGAN, a universal vocoder that generalizes well for various out-of-distribution scenarios without fine-tuning. We introduce periodic activation function and anti-aliased representation into the GAN generator, which brings the desired inductive bias for audio synthesis and significantly improves audio quality. In addition, we train our GAN vocoder at the largest scale up to 112M parameters, which is unprecedented in the literature. We identify and address the failure modes in large-scale GAN training for audio, while maintaining high-fidelity output without over-regularization. Our BigVGAN, trained only on clean speech (LibriTTS), achieves the state-of-the-art performance for various zero-shot (out-of-distribution) conditions, including unseen speakers, languages, recording environments, singing voices, music, and instrumental audio. We release our code and model at: https://github.com/NVIDIA/BigVGAN

Sang-gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, Sungroh Yoon• 2022

Related benchmarks

TaskDatasetResultRank
Speech ReconstructionLibriTTS clean (test)
PESQ4.186
63
Music Source SeparationMUSDB18 HQ (test)
SDR (Drums)4.41
61
Audio ReconstructionMusicDB (test)--
28
Analysis-synthesisMusic Academic
FAD0.013
24
Audio ReconstructionAudioSet (test)
Mel Distance (16kHz)0.363
23
Speech EnhancementSpeech Enhancement (SE) Task (test)
PESQ2.149
22
Speech SynthesisLibriTTS (ID)
PESQ3.6122
20
Waveform GenerationMUSDB18 out-of-distribution vocal samples HQ (test)
M-STFT0.9062
19
Neural VocodingLibriTTS (test)
PESQ4.269
18
Audio GenerationLibriTTS (dev)
M-STFT0.8788
18
Showing 10 of 58 rows

Other info

Code

Follow for update