Bayesian updatings in Hopfield-like associate memory models
Isaac Meilijson · Lecture notes-monograph series · 1996
This article summarizes and explains in statistical terminology two papers written jointly with Eytan Ruppin, presenting a Bayesian outlook on the performance of Hopfield-like attractor neural networks.Restricting attention to the evaluation of performance after two iterations rather than studying thermodynamical limits, we are able to extend the analysis to more general models than those usually considered: input patterns applied to small subsets of neurons, general connectivity architectures of the synaptic network and more efficient use of history.We show that the optimal signal that a Bayesian neuron should emit has a slanted sigmoidal shape as a function of its current field value (or posterior odds), and provide an intuitive account of activation functions with such non-monotone shapes.