Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Masked Language Modeling on Wikipedia + BookCorpus (dev)
Loading...
74.76
MLM Accuracy
RealFormer
61.0424
64.6037
68.165
71.7263
Dec 21, 2020
MLM Accuracy
Updated 4d ago
Evaluation Results
Method
Method
Links
MLM Accuracy
RealFormer
Model=BERT-xLarge, Pre...
2020.12
74.76
RealFormer
Model=BERT-Large, Pre-...
2020.12
73.94
Post-LN
Model=BERT-xLarge, Pre...
2020.12
73.72
Post-LN
Model=BERT-Large, Pre-...
2020.12
73.64
Pre-LN
Model=BERT-xLarge, Pre...
2020.12
73.53
Pre-LN
Model=BERT-Large, Pre-...
2020.12
73.21
RealFormer
Model=BERT-Base, Pre-t...
2020.12
70.42
Post-LN
Model=BERT-Base, Pre-t...
2020.12
70.2
Pre-LN
Model=BERT-Base, Pre-t...
2020.12
69.74
RealFormer
Model=BERT-Small, Pre-...
2020.12
61.7
Pre-LN
Model=BERT-Small, Pre-...
2020.12
61.67
Post-LN
Model=BERT-Small, Pre-...
2020.12
61.57
Feedback
Search any
task
Search any
task