An Adaptive Function Neural Network (ADFUNN) Classifier
Miao Kang, Dominic Palmer-Brown · 2006
ADFUNN is based on a linear piecewise neuron activation function that is modified by a novel gradient descent supervised learning algorithm. It has been applied to some linearly inseparable problems: XOR, Iris dataset, phrase recognition problem. In all cases it exhibited impressive generalisation classification ability with no hidden nodes. In addition, the learned functions support intelligent data analysis. In this paper, we improve the general learning rule of ADFUNN by using proximal proportionality to adapt neural activation functions more accurately. The learned functions are then smoothed in preparation for recognising their closest fit to analytical functions. We compare two different algorithms for smoothing the learned function curves: the simple moving average and least-squares polynomial smoothing. The smoothed curves prove to be accurate replacements for the natural language phrase recognition test case.