Theory of associative memory in randomly connected Boolean neural networks
K. Y. Michael Wong, David C. Sherrington · Journal of Physics A Mathematical and General · 1989
The Aleksander model of neural networks replaces the connection weights of conventional models by logic devices (or Boolean functions). Learning is achieved by adjusting the Boolean functions stepwise via a 'training-with-noise' algorithm. The authors present a theory of the statistical dynamical properties of the randomly connected model and demonstrate that, in the limit of large but dilute connectivity c of the nodes, the storage capacity for associative memory is of the order (2/c 2 )2 c , which corresponds, roughly speaking, to an average of one nearest-neighbouring pattern stored at site distances 2 on each node. Two parameters are introduced into the learning algorithm: q r and q c being respectively the probabilities to register a correct bit and erase an incorrect one. The effects of varying q r , q c and the training noise level on the storage capacity (after very long training) are discussed. In the limit of low training noise level, the training algorithm is equivalent to the so-called 'proximity rules'. Study of its retrieval properties shows that the model can be described as 'short ranged', whereas the Hopfield model is 'long ranged'. The advantages and disadvantages of introducing the intermediate u state into the system are also discussed.