Share your thoughts, 1 month free Claude Pro on usSee more

Language Modeling on WikiText-103 small setting (test)

32.8Perplexity

Delta RNN

Updated 1mo ago

Evaluation Results

Method	Links
Delta RNN 2021.06		32.8
RDN 2021.06		33.6
Delta LSTM 2021.06		33.8
Transformer 2021.06		34.1
Delta RNN 2021.06		35
Delta Net 2021.06		35.2
Delta Delta Net 2021.06		35.2
RDN 2021.06		35.2
Delta MLP 2021.06		36.8
Linear Transformer 2021.06		38.3
NAMEx-Full-Mom 2025.10		82.94
NAMEx-Full 2025.10		83.16
NAMEx-Mom 2025.10		83.59
EP-CAMEx-Mom 2025.10		84.05
NAMEx 2025.10		84.12
CAMEx 2025.10		84.48
SMoE (Top-2) 2025.10		84.81
EP-CAMEx 2025.10		85.03
SMEAR 2025.10		87.24
SMoE (Top-1) 2025.10		87.79