A Bayesian approach to artificial neural network model selection
Greer B. Humphrey, Holger Robert Maier, Martin Francis Lambert · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · 2005
Artificial neural networks (ANNs) have proven to be extremely valuable tools in the field of water resources engineering.However, one of the most difficult tasks in developing an ANN is determining the optimum level of complexity required to model a given problem, as there is no formal systematic model selection method.The generalisability of an ANN, which is defined by its predictive performance on the universe of possible data, can be significantly impaired if there are too few or too many hidden nodes in the network.Therefore, for an ANN to be a valuable prediction tool, it is important that some effort is made to optimise the number of hidden nodes.This paper presents a Bayesian model selection (BMS) method for ANNs that provides an objective approach for comparing models of varying complexity in order to select the most appropriate ANN structure.Given a set of competing models H 1 , . . ., H H , BMS is used to compare the posterior probability that each model H i is the true data generating function, given a set of observed data y.This probability is also known as the evidence of a model and the ratio of two competing models' evidence values, known as the Bayes' factor, can be used to rank the competing models in terms of the relative evidence in support of each model.