Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Causal Prediction on WikiText-2 (val)
Loading...
5.4856
Min Validation Loss
[1,0,0,0]
5.479968
5.517984
5.556
5.594016
Dec 15, 2025
Min Validation Loss
Validation Loss Change (%)
Updated 4d ago
Evaluation Results
Method
Method
Links
Min Validation Loss
Validation Loss Change (%)
[1,0,0,0]
H×L=4×6, T=512, d=256
2025.12
5.4856
0.562
[1,0,0,0]
H×L=16×6, T=512, d=256
2025.12
5.4918
0.108
QKV111
H×L=16×6, T=512, d=256
2025.12
5.4978
0
[1,1,1,1]
H×L=4×6, T=512, d=256
2025.12
5.5088
0.142
QKV111
H×L=4×6, T=512, d=256
2025.12
5.5167
0
[1,1,1,1]
H×L=16×6, T=512, d=256
2025.12
5.5179
-0.364
[0,0,0,0]
H×L=4×6, T=512, d=256
2025.12
5.5524
-0.648
[1,0,0,0]
H×L=1×1, T=512, d=256
2025.12
5.625
0.023
[1,1,1,1]
H×L=1×1, T=512, d=256
2025.12
5.6262
0.002
QKV111
H×L=1×1, T=512, d=256
2025.12
5.6263
0
[0,0,0,0]
H×L=1×1, T=512, d=256
2025.12
5.6264
-0.001
Feedback
Search any
task
Search any
task
