Neural Turing Machines
About
We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.
Alex Graves, Greg Wayne, Ivo Danihelka• 2014
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Natural Language Inference | SNLI (test) | Accuracy81.8 | 681 | |
| Language Modeling | WikiText-103 (test) | Perplexity48.7 | 524 | |
| Sequential Image Classification | PMNIST (test) | Accuracy (Test)90.9 | 77 | |
| Question Answering | bAbI (test) | Mean Error31.42 | 54 | |
| Question Answering | bAbI 10k (test) | Task 1: 1 Supporting Fact Error31.5 | 15 | |
| Copying Task | Copying Task 50 (train) | CE0.00e+0 | 9 | |
| Copying Task | Copying Task 200 (test) | Cross-Entropy2.54 | 9 | |
| Synthetic Copy | Synthetic Copy L=50 (test) | Test Accuracy40.1 | 6 | |
| Synthetic Reverse | Synthetic Reverse L=50 (test) | Test Accuracy61.1 | 6 | |
| Synthetic Copy | Synthetic Copy L=100 (test) | Test Accuracy11.8 | 6 |
Showing 10 of 12 rows