Distributed dynamics in neural networks
Andreas V. M. Herz, Charles M. Marcus · Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics · 1993
We analyze the dynamics and statistical mechanics of attractor neural networks with ``distributed'' updating rules in which groups of one or more neurons are updated simultaneously. Such partially parallel updating schemes are a central feature of neural-network architectures that use many processors, implemented either on special multiprocessor hardware, or among many computers linked over a network. Several updating rules are classified and discussed; these rules generalize the parallel dynamics of the Little model and the one-at-a-time dynamics of the Hopfield model. Analytic results presented herein include a stability criterion that specifies sufficient conditions under which distributed dynamics lead to fixed-point attractors. For binary neurons with block-sequential updating and a Hebbian learning rule, the storage capacity is found as a function of the number of update groups. Several open problems are also discussed.