Dilution in a linear neural network
D M L Barbato, José F. Fontanari · Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics · 1995
The effects of elimination of synaptic weights on the learning capability of a single-layer, feed-forward neural network composed of linear neurons are investigated within the equilibrium statistical mechanics framework of Gardner and co-workers [J. Phys. A 21, 257 (1988); 21, 271 (1988)]. A comparison between the performances of networks damaged by different types of dilution, which may occur either before or after the learning state, shows that the strategy of minimizing the training error does not yield the best generalization performance. Moreover, this comparison also shows that, depending on the size of the training set and on the level of noise corrupting the training data, the smaller weights may become the determinant factors in the good functioning of the network. In particular, the larger the level of noise, the more important the contribution of the smaller weights to the generalization capability of the network.