ON DERIVING DETERMINISTIC LEARNING RULES FROM STOCHASTIC SYSTEMS
Ronny Meir · International Journal of Neural Systems · 1991
We discuss the derivation of deterministic learning rules from an underlying stochastic system. We focus on the symmetrically connected Boltzmann machine and show how various approximations give rise to different learning algorithms. In particular, we show how to derive a symmetrized form of the recurrent back propagation learning algorithm from the Boltzmann machine. We also discuss the connection between the different deterministic learning algorithms focusing on the probability distributions from which they originate. It will also be shown that inspite of the fact that two probability distributions have the same moments to any finite order, they give rise to two distinct learning algorithms.