Our new X account is live! Follow @wizwand_team for updates

Machine Translation on WMT En-De (newstest2014)

31.26BLEU

BiBERT Contextualized Embeddings + Stochastic Layer Selection + Fine-Tuning

Updated 4d ago

Evaluation Results

Method	Links
BiBERT Contextualized Embeddings + Stochastic Layer Selection + Fine-Tuning 2021.09		31.26
BiBERT Contextualized Embeddings + Stochastic Layer Selection 2021.09		30.91
BERT-Fuse 2021.09		30.75
BERT Initialization (12 layers) 2021.09		30.6
BiBERT Contextualized Embeddings + Stochastic Layer Selection + Dual-Directional Training 2021.09		30.31
Evolved Transformer 2021.09		29.8
Transformer + Large Batch 2021.09		29.3
Transformer+ALONE (Real number) 2020.04		27.61
Transformer+ALONE (Binary) 2020.04		27.55
Transformer 2020.04		27.3
Transformer (conventional word embeddings) 2020.04		27.12
Transformer+DeFINE 2020.04		27.01
Transformer+ALONE (Binary) 2020.04		26.97
Transformer+ALONE (Real number) 2020.04		26.95
Transformer+ALONE (Real number) 2020.04		26.93
Transformer+ALONE (Binary) 2020.04		26.9
Transformer+ALONE (Real number) 2020.04		26.85
Transformer+ALONE (Binary) 2020.04		26.75
Transformer (factorized embed) 2020.04		26.56
Transformer (factorized embed) 2020.04		26.43
RL-refined WPM-32K 2016.09		26.3
WPM-32K 2016.09		26.2
ConvS2S 2020.04		25.2
GNMT (RNN Enc-Dec Att) 2016.10		24.61
GNMT 2017.10		24.61
ByteNet 2016.10		23.75
GNMT (RNN Enc-Dec Att) 2016.10		22.62
RNN Enc-Dec Att (Chung et al.) 2016.10		21.33
RNN Enc-Dec Att (Luong et al.) 2016.10		20.9
Phrase Based MT 2016.10		20.7
RNN Enc-Dec Att (Zhou et al.) 2016.10		20.6
RNN Enc-Dec Att (Chung et al.) 2016.10		19.98
Reverse RNN Enc-Dec 2016.10		14
RNN Enc-Dec 2016.10		11.3
Comparable NMT (full parallel) 2017.10		11.05
Proposed (full) + 100k parallel 2017.10		10.95
Proposed (full) + 10k parallel 2017.10		7.86
Proposed (+ BPE) 2017.10		6.89
Proposed (+ backtranslation) 2017.10		6.55
Comparable NMT (100k parallel) 2017.10		5.29
Baseline (emb. nearest neighbor) 2017.10		4.39
Proposed (denoising) 2017.10		2.4
Comparable NMT (10k parallel) 2017.10		0.82