Vigilance Procedure Generalization for Recurrent Associative Memories
Mounir Boukadoum, Sylvain Chartier, Sébastien Hélie, Robert Proulx · eScholarship (California Digital Library) · 2006
Vigilance Procedure Generalization for Recurrent Associative Memories Sylvain Chartier ([email protected]) Centre de recherche de l’Institut Philippe Pinel de Montreal 10,905 Henri-Bourassa Est, Montreal, QC, H1C 1H1, Canada Sebastien Helie ([email protected]) 1 , Robert Proulx ([email protected]) 2 Mounir Boukadoum ([email protected]) 1 Departement d’informatique, Universite du Quebec a Montreal, Departement de psychologie, Universite du Quebec a Montreal POB 8888, station Downtown, Montreal, QC, H3C 3P8, Canada Introduction In our ever changing world, each experienced stimulus differs from the previous. This variation can be explained using two sources: signal noise and exemplars. To overcome the possibly infinite number of stimuli, humans are able to group these unique stimuli into a finite number of categories. In particular, human cognition enables adaptation in many environments, which necessitate a broad range of behaviors which is a function of context. Most unsupervised neural networks cannot deal with such variability. One exception is the family of ART networks, which were proposed to solve the stability - plasticity dilemma (e.g. Carpenter & Grossberg, 1987). These models are able to achieve the desired behavior by using a vigilance procedure. However, this procedure has never been generalized to other classes of unsupervised neural networks, in particular recurrent associative memories. This study proposes a generalization of the vigilance procedure that can be implemented from one-shot binary input learning models (e.g. Hopfield, 1982) to iterative learning real-value patterns models (e.g. Chartier & Proulx, 2005). Vigilance Procedure The role of vigilance is to specify whether a novel stimulus belongs to a previously learned category or a new one. To accomplish this, a new stimulus is shown to the network, and it iterates until convergence. The resulting stable state is compared with the initial stimulus using standard correlation: if the correlation between an initial input (x(0)) and its corresponding attractor x(c) is lower than the vigilance parameter’s value ( ρ ), the new stimulus forms a new category. On the other hand, if the correlation between the stimulus and its corresponding attractor is higher than the vigilance parameter’s value, the new stimulus is integrated into this existing category. In this case, the new stimulus modifies the position of the attractor by using the following average between the initial input and the attractor. x = z ( α x (0) + x ( c ) ) x (0)(1 − z ) 1 + α z where, x is the network’s state used by the given model’s learning rule, α (0 < α << 1) is a parameter which quantifies the effect of the initial input in x and z return 1 if the correlation is greater that ρ and 0 otherwise. Thus, if z = 0, then x = x (0) (initial stimulus); if z = 1, x = ( α x (0) + x (c)) /(1 + α ) (weighted average of the initial and stable states). This procedure is illustrated in Figure 1. Figure 1: Vigilance procedure Conclusion This study shows how to implement a vigilance procedure into RAMs. Consequently, the vigilance procedure is no longer exclusive to competitive networks, which broadens the application domain of RAMs. References Carpenter, G. A. & Grossberg, S. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37, 54-115. Hopfield, J. J.(1982). Neural networks and physical systems with emergent collective computational abilities, Proceedings of the Natural Academy of Sciences (U.S.A.), Chartier, S. & Proulx, R., (2005). NDRAM: Nonlinear Dynamic Recurrent Associative Memory for bipolar and non bipolar learning. IEEE Transactions on Neural Networks, 16, 1393-1400.