Nonmonotonic activation functions in multilayer perceptrons
Gary William Flake · 1993
Multilayer perceptrons (MLPs) and radial basis function networks (RBFNs) are the two most common types of feedforward neural networks used for pattern classification and continuous function approximation. MLPs are characterized by slow learning speed, low memory retention, and small node requirements, while RBFNs are known to have high learning speed, high memory retention, but large node requirements. This dissertation asks and answers the question: "Can we do better?" Two types of neural network architectures are introduced: the hyper-ridge and the hyper-hill. A hyperridge network is a perceptron with no hidden layers and an activation function in the form g(h) = sgn(c 2 \\Gamma h 2 ) (h is the net input; c is a constant "width"), while a hyp...