Sensitivity of madalines to weight errors
Maryhelen Stevenson · 1991
A neural network consists of many interconnected nonlinear processing elements operating in parallel. The processing elements are connected via weights which can be adjusted to realize a desired mapping from input to output. Renewed interest in the field has been sparked by the development of new training algorithms and by recent advances in VLSI technology which can be used to implement large networks. The sensitivity of a neural network to changes in its weights is an important consideration for both the design of hardware and the development of training algorithms for neural networks. In the design of hardware to implement a neural network, it is important to consider the effects of weights with limited precision. Likewise, in the development of training algorithms, it is helpful to understand how finely the weights must be adjusted to achieve precise output objectives. In this dissertation, an analysis on the sensitivity of the Madaline to weight errors is presented. A Madaline is a layered feed-forward neural network of Adaline elements (threshold logic units). An approximation is derived which expresses the probability that for a given input pattern, a selected output of a Madaline changes state as a result of small variations in the weights. This probability of error is expressed as a function of the percentage change in the weights and of the network architecture. As might be expected, the probability of error increases with the number of layers in the network. However, for sufficiently large networks, the probability of error is essentially independent of the number of Adalines per layer. For small networks, the probability of error decreases with decreasing numbers of Adalines per layer.