A Nonlinear Perceptron Algorithm Based on Kernel Functions
Xu Jian · Chinese Journal of Computers · 2002
This paper briefly reviews the perceptron algorithm and introduce its equivalent statement based on inner product. In order to enhance the classification ability of Rosenblatt's perceptron algorithm, authors generalize this algorithm by using kernel idea to yield a nonlinear perceptron algorithm based on kernels, e.g., kernel perceptron algorithm. It combines a simply iterative procedure with kernel functions to fulfill a design of nonlinear classifiers and can deal with the nonlinearly separable problems in the original attribute space and the linearly separable ones in the feature space. For the non separable cases, several heuristic strategies are suggested. Compared with other kernel machines such as SVM, KFD and KPCA, the algorithm structure of kernel perceptron algorithms is the simplest. This paper also analyzes the relation between the algorithm and radial basis function network, potential function method and support vector machine in detail. In the experiment aspects, the results of two artificial data and two benchmark databases are reported and analyzed. For the linear example, the classical and kernel perceptron methods both find the separated hyperplanes, which can classify all samples. About two spirals problem, the nonlinear decision plane obtained by kernel perceptron with radial basis function kernel can separate all samples lying in two spirals. For the image segmentation data, the correct rates of linear and kernel perceptron algorithm are 74.0% and 90.76% respectively. Since there exist 100 realizations in thyroid data set, the average error rate and variation from the linear perceptron are 14.23% and 5.74% respectively, while those from kernel one are 4.65% and 2.39%. Such experiment results show that kernel perceptron algorithm effectively improves the classification precision compared with the linear one.