Evolution programs for Bayesian training of neural networks
Lemuel R. Myers, John G. Keller, Steven K. Rogers, Matthew Kabrisky · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1998
In this paper, it is shown that Evolution Programs can be used to search the weight space for Bayesian training of a Neural Network. Bayesian Analysis is an integration problem (as opposed to an optimization problem) over weight space. The first application of the Bayesian method primarily focused on using a Gaussian approximation of the posterior distribution in an area of high probability in the weight space instead of using formal integration. More recently, training a neural network in a Bayesian fashion has been accomplished by searching weight space for areas of high probability density which obviates the need for the Gaussian assumption. In particular, a hybrid Monte-Carlo method was used to search weight space in a logical manner to obtain an arbitrarily close approximation of the integration involved in a Bayesian analysis. Genetic Algorithms have been used in the past to determine the weights in an ANN, and (with some slight modifications) are ideally suited for searching the weight space to approximate the Bayesian integration. In this respect, the Bayesian framework provides a simple and elegant way to apply Evolution Programs to the ANN training problem. While this paper concentrates on using ANNs as classifiers, the generalization to regression problems is straightforward.