Dynamics of learning in recurrent feature-discovery Networks
Todd K. Leen · 2018
The self-organization of recurrent feature-discovery networks is studied from the perspective of dynamical systems. Bifurcation theory reveals parameter regimes in which multiple equilibria or limit cycles coexist with the equilibrium at which the networks perform principal component analysis. 1 Introduction Oja (1982) made the remarkable observation that a simple model neuron with an Hebbian adaptation rule develops into a filter for the first principal component of the input distribution. Several researchers have extended Oja's work, developing networks that perform a complete principal component analysis (PCA). Sanger (1989) proposed an algorithm that uses a single layer of weights with a set of cascaded feedback projections to force nodes to filter for the principal components. This architecture singles out a particular node for each principal component. Oja (1989) and Oja and Karhunen (1985) give a related algorithm that projects inputs onto an orthogonal basis spanning the princ...