Reservoir Stack Machines

Paaßen B, Schulz A, Hammer B (2021)
Neurocomputing 470: 352-364.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Abstract / Bemerkung
Memory-augmented neural networks equip a recurrent neural network with an explicit memory to support tasks that require information storage without interference over long times. A key motivation for such research is to perform classic computation tasks, such as parsing. However, memory-augmented neural networks are notoriously hard to train, requiring many backpropagation epochs and a lot of data. In this paper, we introduce the reservoir stack machine, a model which can provably recognize all deterministic context-free languages and circumvents the training problem by training only the output layer of a recurrent net and employing auxiliary information during training about the desired interaction with a stack. In our experiments, we validate the reservoir stack machine against deep and shallow networks from the literature on three benchmark tasks for Neural Turing machines and six deterministic context-free languages. Our results show that the reservoir stack machine achieves zero error, even on test sequences longer than the training data, requiring only a few seconds of training time and 100 training sequences.
Stichworte
reservoir computing; neural turing machines; pushdown automata; context-free grammars; echo state networks
Erscheinungsjahr
2021
Zeitschriftentitel
Neurocomputing
Band
470
Seite(n)
352-364
ISSN
0925-2312
Page URI
https://pub.uni-bielefeld.de/record/2954542

Zitieren

Paaßen B, Schulz A, Hammer B. Reservoir Stack Machines. Neurocomputing. 2021;470:352-364.
Paaßen, B., Schulz, A., & Hammer, B. (2021). Reservoir Stack Machines. Neurocomputing, 470, 352-364. https://doi.org/10.1016/j.neucom.2021.05.106
Paaßen, Benjamin, Schulz, Alexander, and Hammer, Barbara. 2021. “Reservoir Stack Machines”. Neurocomputing 470: 352-364.
Paaßen, B., Schulz, A., and Hammer, B. (2021). Reservoir Stack Machines. Neurocomputing 470, 352-364.
Paaßen, B., Schulz, A., & Hammer, B., 2021. Reservoir Stack Machines. Neurocomputing, 470, p 352-364.
B. Paaßen, A. Schulz, and B. Hammer, “Reservoir Stack Machines”, Neurocomputing, vol. 470, 2021, pp. 352-364.
Paaßen, B., Schulz, A., Hammer, B.: Reservoir Stack Machines. Neurocomputing. 470, 352-364 (2021).
Paaßen, Benjamin, Schulz, Alexander, and Hammer, Barbara. “Reservoir Stack Machines”. Neurocomputing 470 (2021): 352-364.

Link(s) zu Volltext(en)
Access Level
OA Open Access

Software:
Beschreibung
Software implementation and experimental code
Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Web of Science

Dieser Datensatz im Web of Science®
Quellen

arXiv: 2105.01616

Suchen in

Google Scholar