Learning in feedforward networks with nonsmooth functions: an I/sub infinity / example
N.J. Redding, Tom Downs · 1991
The authors consider the problem of learning in networks where some or all of the functions involved are not smooth. Examples of such networks are those whose neural transfer functions are piecewise-linear and those whose error function is defined in terms of the I/sub infinity / norm. The authors draw upon some results from the field of nonsmooth optimization (NSO) to present an algorithm for the nonsmooth case. They demonstrate the viability of using NSO for training networks in cases that standard procedures, with their implicit smoothness assumption, would find difficult or impossible. The motivation for this work arose out of the fact that it has been possible to show that an error function based on the I/sub infinity / norm overcomes the difficulties which can occur when using backpropagation's I/sub 2/ norm.>