# Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions

Wersing H, Beyn W-J, Ritter H (2001)

Neural Computation 13(8): 1811-1825.

*Journal Article*|

*Published*|

*English*

No fulltext has been uploaded

Department

Abstract

We establish two conditions that ensure the nondivergence of additive recurrent networks with unsaturating piecewise linear transfer functions, also called linear threshold or semilinear transfer functions. As Hahnloser, Sarpeshkar, Mahowald, Douglas, and Seung (2000) showed, networks of this type can be efficiently built in silicon and exhibit the coexistence of digital selection and analog amplification in a single circuit. To obtain this behavior, the network must be multistable and nondivergent, and our conditions allow determining the regimes where this can be achieved with maximal recurrent amplification. The first condition can be applied to nonsymmetric networks and has a simple interpretation of requiring that the strength of local inhibition match the sum over excitatory weights converging onto a neuron. The second condition is restricted to symmetric networks, but can also take into account the stabilizing effect of nonlocal inhibitory interactions. We demonstrate the application of the conditions on a simple example and the orientation-selectivity mo del of Ben-Yishai, Lev Bar-Or, and Sompolinsky (1995). We show that the conditions can be used to identify in their model regions of maximal orientation-selective amplification and symmetry breaking.

Publishing Year

ISSN

PUB-ID

### Cite this

Wersing H, Beyn W-J, Ritter H. Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions.

*Neural Computation*. 2001;13(8):1811-1825.Wersing, H., Beyn, W. - J., & Ritter, H. (2001). Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions.

*Neural Computation*,*13*(8), 1811-1825.Wersing, H., Beyn, W. - J., and Ritter, H. (2001). Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions.

*Neural Computation*13, 1811-1825.Wersing, H., Beyn, W.-J., & Ritter, H., 2001. Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions.

*Neural Computation*, 13(8), p 1811-1825. H. Wersing, W.-J. Beyn, and H. Ritter, “Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions”,

*Neural Computation*, vol. 13, 2001, pp. 1811-1825. Wersing, H., Beyn, W.-J., Ritter, H.: Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation. 13, 1811-1825 (2001).

Wersing, Heiko, Beyn, Wolf-Jürgen, and Ritter, Helge. “Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions”.

*Neural Computation*13.8 (2001): 1811-1825.
This data publication is cited in the following publications:

This publication cites the following data publications:

### 25 Citations in Europe PMC

Data provided by Europe PubMed Central.

Scale-limited activating sets and multiperiodicity for threshold-linear networks on time scales.

Huang Z, Raffoul YN, Cheng CY.,

PMID: 23757562

Huang Z, Raffoul YN, Cheng CY.,

*IEEE Trans Cybern*44(4), 2014PMID: 23757562

Boundedness and complete stability of complex-valued neural networks with time delay.

Bo Zhou , Qiankun Song .,

PMID: 24808563

Bo Zhou , Qiankun Song .,

*IEEE Trans Neural Netw Learn Syst*24(8), 2013PMID: 24808563

Persistence and storage of activity patterns in spiking recurrent cortical networks: modulation of sigmoid signals by after-hyperpolarization currents and acetylcholine.

Palma J, Grossberg S, Versace M.,

PMID: 22754524

Palma J, Grossberg S, Versace M.,

*Front Comput Neurosci*6(), 2012PMID: 22754524

After-hyperpolarization currents and acetylcholine control sigmoid transfer functions in a spiking cortical model.

Palma J, Versace M, Grossberg S.,

PMID: 21779754

Palma J, Versace M, Grossberg S.,

*J Comput Neurosci*32(2), 2012PMID: 21779754

Memory dynamics in attractor networks with saliency weights.

Tang H, Li H, Yan R.,

PMID: 20235821

Tang H, Li H, Yan R.,

*Neural Comput*22(7), 2010PMID: 20235821

Foundations of implementing the competitive layer model by Lotka-Volterra recurrent neural networks.

Yi Z.,

PMID: 20142165

Yi Z.,

*IEEE Trans Neural Netw*21(3), 2010PMID: 20142165

Nontrivial global attractors in 2-D multistable attractor neural networks.

Zou L, Tang H, Tan KC, Zhang W.,

PMID: 19884069

Zou L, Tang H, Tan KC, Zhang W.,

*IEEE Trans Neural Netw*20(11), 2009PMID: 19884069

Multistability and new attraction basins of almost-periodic solutions of delayed neural networks.

Wang L, Lu W, Chen T.,

PMID: 19709974

Wang L, Lu W, Chen T.,

*IEEE Trans Neural Netw*20(10), 2009PMID: 19709974

State-dependent computation using coupled recurrent networks.

Rutishauser U, Douglas RJ.,

PMID: 19431267

Rutishauser U, Douglas RJ.,

*Neural Comput*21(2), 2009PMID: 19431267

Permitted and forbidden sets in discrete-time linear threshold recurrent neural networks.

Yi Z, Zhang L, Yu J, Tan KK.,

PMID: 19423436

Yi Z, Zhang L, Yu J, Tan KK.,

*IEEE Trans Neural Netw*20(6), 2009PMID: 19423436

Representations of continuous attractors of recurrent neural networks.

Yu J, Yi Z, Zhang L.,

PMID: 19150791

Yu J, Yi Z, Zhang L.,

*IEEE Trans Neural Netw*20(2), 2009PMID: 19150791

Analysis of continuous attractors for 2-D linear threshold neural networks.

Zou L, Tang H, Tan KC, Zhang W.,

PMID: 19129036

Zou L, Tang H, Tan KC, Zhang W.,

*IEEE Trans Neural Netw*20(1), 2009PMID: 19129036

Multiperiodicity and attractivity of delayed recurrent neural networks with unsaturating piecewise linear transfer functions.

Zhang L, Yi Z, Yu J.,

PMID: 18269947

Zhang L, Yi Z, Yu J.,

*IEEE Trans Neural Netw*19(1), 2008PMID: 18269947

Selectivity and stability via dendritic nonlinearity.

Morita K, Okada M, Aihara K.,

PMID: 17521280

Morita K, Okada M, Aihara K.,

*Neural Comput*19(7), 2007PMID: 17521280

Learning lateral interactions for feature binding and sensory segmentation from prototypic basis interactions.

Weng S, Wersing H, Steil JJ, Ritter H.,

PMID: 16856650

Weng S, Wersing H, Steil JJ, Ritter H.,

*IEEE Trans Neural Netw*17(4), 2006PMID: 16856650

Dynamics analysis and analog associative memory of networks with LT neurons.

Tang H, Tan KC, Teoh EJ.,

PMID: 16566468

Tang H, Tan KC, Teoh EJ.,

*IEEE Trans Neural Netw*17(2), 2006PMID: 16566468

Analysis of cyclic dynamics for networks of linear threshold neurons.

Tang HJ, Tan KC, Zhang W.,

PMID: 15563749

Tang HJ, Tan KC, Zhang W.,

*Neural Comput*17(1), 2005PMID: 15563749

Discrimination networks for maximum selection.

Jain BJ, Wysotzki F.,

PMID: 14690714

Jain BJ, Wysotzki F.,

*Neural Netw*17(1), 2004PMID: 14690714

Multistability of discrete-time recurrent neural networks with unsaturating piecewise linear activation functions.

Yi Z, Tan KK.,

PMID: 15384526

Yi Z, Tan KK.,

*IEEE Trans Neural Netw*15(2), 2004PMID: 15384526

Analyzing stability of equilibrium points in neural networks: a general approach.

Truccolo WA, Rangarajan G, Chen Y, Ding M.,

PMID: 14622876

Truccolo WA, Rangarajan G, Chen Y, Ding M.,

*Neural Netw*16(10), 2003PMID: 14622876

Multistability analysis for recurrent neural networks with unsaturating piecewise linear transfer functions.

Yi Z, Tan KK, Lee TH.,

PMID: 12620161

Yi Z, Tan KK, Lee TH.,

*Neural Comput*15(3), 2003PMID: 12620161

Permitted and forbidden sets in symmetric threshold-linear networks.

Hahnloser RH, Seung HS, Slotine JJ.,

PMID: 12620160

Hahnloser RH, Seung HS, Slotine JJ.,

*Neural Comput*15(3), 2003PMID: 12620160

Selectively grouping neurons in recurrent networks of lateral inhibition.

Xie X, Hahnloser RH, Seung HS.,

PMID: 12433293

Xie X, Hahnloser RH, Seung HS.,

*Neural Comput*14(11), 2002PMID: 12433293

Dynamic stability conditions for Lotka-Volterra recurrent neural networks with delays.

Yi Z, Tan KK.,

PMID: 12241387

Yi Z, Tan KK.,

*Phys Rev E Stat Nonlin Soft Matter Phys*66(1 Pt 1), 2002PMID: 12241387

Attentional recruitment of inter-areal recurrent networks for selective gain control.

Hahnloser RH, Douglas RJ, Hepp K.,

PMID: 12079551

Hahnloser RH, Douglas RJ, Hepp K.,

*Neural Comput*14(7), 2002PMID: 12079551

### 18 References

Data provided by Europe PubMed Central.

On the piecewise analysis of networks of linear threshold neurons.

Hahnloser RL.,

PMID: 12662807

Hahnloser RL.,

*Neural Netw*11(4), 1998PMID: 12662807

A model for the depth-dependence of receptive field size and contrast sensitivity of cells in layer 4C of macaque striate cortex.

Bauer U, Scholz M, Levitt JB, Obermayer K, Lund JS.,

PMID: 10341989

Bauer U, Scholz M, Levitt JB, Obermayer K, Lund JS.,

*Vision Res.*39(3), 1999PMID: 10341989

A model for the intracortical origin of orientation preference and tuning in macaque striate cortex.

Adorjan P, Levitt JB, Lund JS, Obermayer K.,

PMID: 10367965

Adorjan P, Levitt JB, Lund JS, Obermayer K.,

*Vis. Neurosci.*16(2), 1999PMID: 10367965

Recurrent excitation in neocortical circuits.

Douglas RJ, Koch C, Mahowald M, Martin KA, Suarez HH.,

PMID: 7638624

Douglas RJ, Koch C, Mahowald M, Martin KA, Suarez HH.,

*Science*269(5226), 1995PMID: 7638624

Piecewise affine bijections of ?n, and the equation Sx+−Tx−=y

Kuhn,

Kuhn,

*Linear Algebra and its Applications*96(1), 1987
A competitive-layer model for feature binding and sensory segmentation.

Wersing H, Steil JJ, Ritter H.,

PMID: 11177439

Wersing H, Steil JJ, Ritter H.,

*Neural Comput*13(2), 2001PMID: 11177439

On Neurodynamics with Limiter Function and Linsker's Developmental Model

Feng,

Feng,

*Neural Computation*8(5), 1996Lippmann,

*IEEE ASSP Magazine*4(2), 1987

Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit.

Hahnloser RH, Sarpeshkar R, Mahowald MA, Douglas RJ, Seung HS.,

PMID: 10879535

Hahnloser RH, Sarpeshkar R, Mahowald MA, Douglas RJ, Seung HS.,

*Nature*405(6789), 2000PMID: 10879535

A model of multiplicative neural responses in parietal cortex.

Salinas E, Abbott LF.,

PMID: 8876244

Salinas E, Abbott LF.,

*Proc. Natl. Acad. Sci. U.S.A.*93(21), 1996PMID: 8876244

Computational differences between asymmetrical and symmetrical networks.

Li Z, Dayan P.,

PMID: 10372762

Li Z, Dayan P.,

*Network*10(1), 1999PMID: 10372762

Feng,

*Journal of Physics A Mathematical and General*29(16), 1996

Convergent activation dynamics in continuous time networks

Hirsch,

Hirsch,

*Neural Networks*2(5), 1989
Equilibria of the brain-state-in-a-box (BSB) neural model

Greenberg,

Greenberg,

*Neural Networks*1(4), 1988
Spatial summation of inhibitory influences in the eye of Limulus, and the mutual interaction of receptor units.

HARTLINE HK, RATLIFF F.,

PMID: 13525682

HARTLINE HK, RATLIFF F.,

*J. Gen. Physiol.*41(5), 1958PMID: 13525682

Self-organization of orientation sensitive cells in the striate cortex.

von der Malsburg C.,

PMID: 4786750

von der Malsburg C.,

*Kybernetik*14(2), 1973PMID: 4786750

Lyapunov functions for neural nets with nondifferentiable input-output characteristics.

Feng J.,

PMID: 9117900

Feng J.,

*Neural Comput*9(1), 1997PMID: 9117900

Theory of orientation tuning in visual cortex.

Ben-Yishai R, Bar-Or RL, Sompolinsky H.,

PMID: 7731993

Ben-Yishai R, Bar-Or RL, Sompolinsky H.,

*Proc. Natl. Acad. Sci. U.S.A.*92(9), 1995PMID: 7731993

### Export

0 Marked Publications### Web of Science

View record in Web of Science®### Sources

PMID: 11506671

PubMed | Europe PMC