Function Approximation with the Sweeping Hinge Algorithm
Don R. Hush, Fernando Lozano, Bill G. Horne · Neural Information Processing Systems · 1997
We present a computationally efficient algorithm for function approximation with piecewise linear sigmoidal nodes. A one hidden layer network is constructed one node at a time using the method of fitting the residual. The task of fitting individual nodes is accomplished using a new algorithm that searchs for the best fit by solving a sequence of Quadratic Programming problems. This approach offers significant advantages over derivative-based search algorithms (e.g. backpropagation and its extensions). Unique characteristics of this algorithm include: finite step convergence, a simple stopping criterion, a deterministic methodology for seeking local minima, good scaling properties and a robust numerical implementation.