Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Stable Audio Open

About

Open generative models are vitally important for the community, allowing for fine-tunes and serving as baselines when presenting new models. However, most current text-to-audio models are private and not accessible for artists and researchers to build upon. Here we describe the architecture and training process of a new open-weights text-to-audio model trained with Creative Commons data. Our evaluation shows that the model's performance is competitive with the state-of-the-art across various metrics. Notably, the reported FDopenl3 results (measuring the realism of the generations) showcase its potential for high-quality stereo sound synthesis at 44.1kHz.

Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, Jordi Pons• 2024

Related benchmarks

TaskDatasetResultRank
Text-to-Audio GenerationAudioCaps (test)
FAD2.32
138
Audio ReconstructionSong Describer
L/R Mel0.6863
10
Context Length EstimationSong Describer
Context Length (s)106
10
Text-to-AudioAudioSet Strong
F1 Event6.05
9
Continuous Audio Compression48 kHz Sound Effects
FAD0.78
7
Text-to-AudioText-to-Audio (test)
Loudness MAE17.49
7
Music GenerationSong Describer Dataset (test)
FDopenl396.51
5
Text-to-AudioAudioCaps multi-event prompts
FDopenl388.5
5
Text-to-Music GenerationSong Describer Dataset (full)
FD_openl399.7
5
Text-to-Audio GenerationHuman Evaluation Subjective Audio Assessment (test)
Z-Score (OVL)0.0723
4
Showing 10 of 12 rows

Other info

Code

Follow for update