Our new X account is live! Follow @wizwand_team for updates
Home
/
Benchmarks
Language Modeling on 100 Billion Word Google News Dataset (test)
Loading...
38.2
Test Perplexity (0.1 epochs)
MoE-16384-h
37.044
44.847
52.65
60.453
Jan 23, 2017
Test Perplexity (0.1 epochs)
Test Perplexity (1 epoch)
Updated 4d ago
Evaluation Results
Method
Method
Links
Test Perplexity (0.1 epochs)
Test Perplexity (1 epoch)
MoE-16384-h
ops/timestep (millions...
2017.01
38.2
29.7
MoE-65536-h
ops/timestep (millions...
2017.01
38.2
28.9
MoE-4096-h
ops/timestep (millions...
2017.01
38.9
30.9
MoE-131072-h
ops/timestep (millions...
2017.01
39.8
29.2
MoE-1024-h
ops/timestep (millions...
2017.01
40.3
32.7
MoE-256-h
ops/timestep (millions...
2017.01
42.8
35.3
MoE-32
ops/timestep (millions...
2017.01
48.5
40.4
4xLSTM-512
ops/timestep (millions...
2017.01
54.5
47
Kneser-Ney 5-gram
ops/timestep (millions...
2017.01
67.1
45.3
Feedback
Search any
task
Search any
task