SOTA Character-level Language Modeling on Hutter Prize Wikipedia (test) and PapersWithCode

1.08Bits/Char

mLSTM

Updated 4d ago

Evaluation Results

Method
mLSTM 2017.09	1.08	-
mLSTM 2017.09	1.13	-
large mLSTM +emb +WN +VD 2016.09	1.24	-
mLSTM 2017.09	1.24	-
Fast-slow LSTM (rec depth 4) + zoneout 2016.09	1.25	-
Fast-slow LSTM 2017.09	1.25	-
RHN (rec depth 7) + VD 2016.09	1.27	-
Recurrent highway networks 2017.09	1.27	-
mLSTM +emb +WN +VD 2016.09	1.28	-
LSTM (4 layer) + VD + BB tuning 2016.09	1.3	-
LSTM + BB tuning 2017.09	1.3	-
bytenet decoder 2016.09	1.31	-
Bytenet decoder 2017.09	1.31	-
hierarchical multiscale LSTM 2016.09	1.32	-
Hierarchical multiscale LSTM 2017.09	1.32	-
stacked LSTM (7-layer) + dynamic eval 2016.09	1.33	-
Stacked LSTM 2017.09	1.33	-
hyperLSTM 2016.09	1.34	-
HyperLSTM 2017.09	1.34	-
feedback LSTM + zoneout 2016.09	1.37	-
recurrent memory array structures 2016.09	1.4	-
unregularised mLSTM 2016.09	1.4	-
unregularised mLSTM (RMS prop, 4 epoch) 2016.09	1.42	-
MI-LSTM 2016.09	1.44	-
mLSTM +emb +WN 2016.09	1.44	-
Multiplicative integration LSTM 2017.09	1.44	-
stacked LSTM (7-layer) 2016.09	1.67	-
Stacked LSTM 2017.09	1.67	-
Stacked LSTM 2016.10	-	1.67
GF-LSTM 2016.10	-	1.58
Grid-LSTM 2016.10	-	1.47
Layer-normalized LSTM 2016.10	-	1.46
MI-LSTM 2016.10	-	1.44
Recurrent Memory Array Structures 2016.10	-	1.4
HM-LSTM 2016.10	-	1.4
Layer Norm HyperLSTM 2016.10	-	1.38
Large Layer Norm HyperLSTM 2016.10	-	1.34
Recurrent Highway Networks 2016.10	-	1.32
ByteNet Decoder 2016.10	-	1.31