Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Learning Stochastic Recurrent Networks

About

Leveraging advances in variational inference, we propose to enhance recurrent neural networks with latent variables, resulting in Stochastic Recurrent Networks (STORNs). The model i) can be trained with stochastic gradient methods, ii) allows structured and multi-modal conditionals at each time step, iii) features a reliable estimator of the marginal likelihood and iv) is a generalisation of deterministic recurrent neural networks. We evaluate the method on four polyphonic musical data sets and motion capture data.

Justin Bayer, Christian Osendorfer• 2014

Related benchmarks

TaskDatasetResultRank
Polyphonic music modelingNottingham (Nott)
NLL (nats)2.85
14
Polyphonic music modelingJSB Chorales
Negative Log-Likelihood (nats)6.91
14
Polyphonic music modelingPiano-midi.de
NLL (nats)7.13
12
Polyphonic music modelingMuseData (Muse)
Negative Log-Likelihood (nats)6.16
12
Polyphonic Music GenerationNottingham (test)
NLL2.85
11
Polyphonic Music GenerationPiano (test)
Negative Log-Likelihood7.13
5
Polyphonic Music GenerationMusedata (test)
NLL6.16
5
Polyphonic Music GenerationJSB (test)
Negative Log-Likelihood6.91
5
Showing 8 of 8 rows

Other info

Follow for update