Comparison of performance of variants of single-layer perceptron algorithms on nonseparable data
Ruchi Parekh · 2000
We present a detailed experimental comparison of the pocket algorithm, thermal perceptron, and barycentric correction procedure algorithms that most commonly used algorithms for training threshold logic units (TLUs). Each of these algorithms represent stable variants of the standard perceptron learning rule in that they guarantee convergence to zero classi cation errors on datasets that are linearly separable and attempt to classify as large a subset of the training patterns as possible for datasets that are not linearly separable. For datasets involving patterns distributed among M di erent categories (M>2) a group of M TLUs is trained, one for each of the output classes. These TLU's can be trained either independently or as a winner-take-all (WTA) group. The latter mechanism accounts for the interactions among the di erent output classes and exploits the fact that a pattern can ideally belong to only one of the M output classes. The extension of the pocket algorithm to the WTA output strategy is direct. In this paper we present heuristic extensions of the thermal perceptron and the barycentric correction procedure to WTA groups and empirically verify their performance. The performance of these algorithms was measured in a collection of carefully chosen benchmarks datasets. We report the training and generalization accuracies of these algorithms on the di erent datasets along with the learning time in seconds. In addition, a comparison of the learning speeds of the algorithms is indicated by means of learning curve plots on two datasets. We identify and report some distinguishing traits of these algorithms which could possibly enable making an informed choice of the training algorithm (combined with constructive learning algorithms) when certain characteristics of the dataset are known. 1