A Mean Field Algorithm for Bayes Learning in Large Feed-forward Neural Networks
Manfred Opper, Ole Winther · 1996
We present a mean field algorithm to realise Bayes optimal predictions in large feed-forward networks. The derivation of the algorithm is based on methods developed within statistical mechanics of disordered systems. The algorithm also provides a leave-one-out cross-validation test of the predictions. Simulations show excellent agreement with theoretical results of statistical mechanics. Furthermore we derive other simpler mean field algorithms and make a comparison study. Category: Theory, statistical mechanics Oral presentation preferred Correspondence should be adressed to Manfred Opper 1 INTRODUCTION Bayes methods have become popular as a consistent framework for regularization and model selection in the field of neural networks (see e.g. [1]). In the Bayes approach to statistical inference [2] one assumes that the prior uncertainty about parameters of an unknown data generating mechanism can be encoded in a probability distribution, the so called prior. Using the prior and the ...