A study of single-hidden-layer perceptrons
Margarita Dorotea Takach, David L. Johnson · 1990
An investigation into the behavior and characteristics of multi-layer perceptrons is presented. The approach used in the investigation considered multi-year perceptrons as a form of function-fitter. This function-fitting model applies for both classification and continuous function approximation problems. What follows from the function-fitting model is that the abilities of multi-layer perceptrons to function-fit to data are determined by the training data and the similarity in behavior between the network architecture and the system approximated. It also follows that the properties of fault tolerance,and noise and outlier immunity are not necessarily superior to those of other conventional approximating techniques. These properties, their relationship to the network architecture and the training data, and the function-fitting abilities of multi-layer perceptrons with a single hidden layer are experimentally explored. The function-fitting model also suggests the mechanisms by which multi-layer perceptrons approximate continuous and discrete valued functions. These mechanisms are analytically and experimentally investigated. A geometric interpretation of the behavior of hidden nodes is used to develop a specific process for estimating the connection values in a network. These estimates can significantly speed up convergence to a solution. Finally, the factors that determine the minimum number of hidden nodes required for a problem are explored through a study of the role of hidden nodes in a network approximation.