Deep Learning in an Adaptive Function Neural Network
Dominic Palmer-Brown, Miao Kang · UEL Research Repository (University of East London) · 2006
Artificial neural network learning is typically accomplished via adaptation between neurons. This paper describes adaptation that is simultaneously between and within neurons. The conventional neurocomputing wisdom is that by adapting the pattern of connections between neurons the network can learn to respond differentially to classes of incoming patterns. The success of this approach in an age of massively increasing computing power that has made high speed neurocomputing feasible on the desktop and more recently in the palm of the hand, has resulted in little attention being paid to the implications of adaptation within the individual neurons. The computational assumption has tended to be that the internal neural mechanism is fixed. However, there are good computational and biological reasons for examining the internal neural mechanisms of learning. Recent neuroscience suggests that neuromodulators play a role in learning by modifying the neuron’s activation function [Scheler] and with an adaptive function approach it is possible to learn linearly inseparable problems fast, even without hidden nodes. The ADaptive FUction Neural Network (ADFUNN) presented in this paper is based on a linear piecewise neuron activation function that is modified by a novel gradient descent supervised learning algorithm [Palmer-Brown;Kang]. It has been applied to the Iris dataset, and a natural language phrase recognition problem, exhibiting impressive generalisation classification ability with no hidden neurons.