The Kernel Adaline: A New Algorithm for Non-Linear Signal Processing and Regression
Thilo-Thomas Frieb, Robert F. Harrison · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · 1998
A certain class of non-linear algorithm for signal processing and machine learning is based on the same intrinsic principle. Some given samples (training points) are in the first stage mapped into a very high-dimensional LINEARISATION space (the feature space of pattern recognition theory) and then a linear algorithm performs its work in this space. The expensive expansion into the linearisation space can be performed efficiently using Mercer's kernel functions, as studied by Aizerman et al. in the 1960's. In this work a non-linear adaptation of the (so far linear) Adaline algorithm by Widrow and Hoff is proposed. The new algorithm combines the conceptual simplicity of a least mean square algorithm for linear regression but exhibits the power of a universal non-linear function to approximator. The kernel Adaline algorithm is introduced and the first experimental results are given.