Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Character-level language modeling on text8
Loading...
0.98
BPC
GPT2
0.9612
1.0881
1.215
1.3419
Mar 4, 2018
Jan 15, 2019
Nov 29, 2019
Oct 12, 2020
Aug 26, 2021
Jul 10, 2022
May 24, 2023
BPC
Updated 4d ago
Evaluation Results
Method
Method
Links
BPC
GPT2
#params=1542M
2023.05
0.98
Focus
#params=22M
2023.05
0.98
24L Transformer-XL
Number of Parameters=277M
2019.01
1.08
Transformer XL
#params=277M
2023.05
1.08
Focus-H (ablation)
#params=21M
2023.05
1.1
64L Transformer
Number of Parameters=235M
2019.01
1.13
12L Transformer
Number of Parameters=44M
2019.01
1.18
Transformer-XL 24B
Architecture=(sf)x12,...
2020.09
1.18
Sandwich Transformer 24B
Architecture=(s)x3 (sf...
2020.09
1.18
PAR Transformer 24B
Architecture=(sff)x5 (...
2020.09
1.18
RHN
Number of Parameters=45M
2019.01
1.27
Large mLSTM
Number of Parameters=45M
2019.01
1.27
LN HM-LSTM
Number of Parameters=35M
2019.01
1.29
HM-LSTM
Size=>12M
2018.03
1.29
BN-LSTM
2019.01
1.36
TCN
Size=4.6M
2018.03
1.45
Feedback
Search any
task
Search any
task