Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods

Neumann K, Emmerich C, Steil JJ (2012)
Journal of Intelligent Learning Systems and Applications 4(3): 230-246.

Zeitschriftenaufsatz | Veröffentlicht | Englisch
 
Download
OA
Abstract / Bemerkung
Neural networks based on high-dimensional random feature generation have become popular under the notions extreme learning machine (ELM) and reservoir computing (RC). We provide an in-depth analysis of such networks with respect to feature selection, model complexity, and regularization. Starting from an ELM, we show how recurrent connections increase the effective complexity leading to reservoir networks. On the contrary, intrinsic plasticity (IP), a biologically inspired, unsupervised learning rule, acts as a task-specific feature regularizer, which tunes the effective model complexity. Combing both mechanisms in the framework of static reservoir computing, we achieve an excellent balance of feature complexity and regularization, which provides an impressive robustness to other model selection parameters like network size, initialization ranges, or the regularization parameter of the output learning. We demonstrate the advantages on several synthetic data as well as on benchmark tasks from the UCI repository providing practical insights how to use high-dimensional random networks for data processing
Stichworte
Extreme Learning Machine; Reservoir Computing; Model Selection; Feature Selection; Model Complexity; Intrinsic Plasticity; Regularization
Erscheinungsjahr
2012
Zeitschriftentitel
Journal of Intelligent Learning Systems and Applications
Band
4
Ausgabe
3
Seite(n)
230-246
ISSN
2150-8402
eISSN
2150-8410
Finanzierungs-Informationen
Article Processing Charge funded by the Deutsche Forschungsgemeinschaft and the Open Access Publication Fund of Bielefeld University.
Page URI
https://pub.uni-bielefeld.de/record/2508614

Zitieren

Neumann K, Emmerich C, Steil JJ. Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods. Journal of Intelligent Learning Systems and Applications. 2012;4(3):230-246.
Neumann, K., Emmerich, C., & Steil, J. J. (2012). Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods. Journal of Intelligent Learning Systems and Applications, 4(3), 230-246. doi:10.4236/jilsa.2012.43024
Neumann, K., Emmerich, C., and Steil, J. J. (2012). Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods. Journal of Intelligent Learning Systems and Applications 4, 230-246.
Neumann, K., Emmerich, C., & Steil, J.J., 2012. Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods. Journal of Intelligent Learning Systems and Applications, 4(3), p 230-246.
K. Neumann, C. Emmerich, and J.J. Steil, “Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods”, Journal of Intelligent Learning Systems and Applications, vol. 4, 2012, pp. 230-246.
Neumann, K., Emmerich, C., Steil, J.J.: Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods. Journal of Intelligent Learning Systems and Applications. 4, 230-246 (2012).
Neumann, Klaus, Emmerich, Christian, and Steil, Jochen J. “Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods”. Journal of Intelligent Learning Systems and Applications 4.3 (2012): 230-246.
Alle Dateien verfügbar unter der/den folgenden Lizenz(en):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Volltext(e)
Access Level
OA Open Access
Zuletzt Hochgeladen
2019-09-06T09:18:03Z
MD5 Prüfsumme
e5d7d4b6877f45fe14687b36d3c267af

Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Suchen in

Google Scholar