A new supervised local modelling classifier based on information theory
David Martínez‐Rego, Óscar Fontenla-Romero, Iago Porto-Díaz, Amparo Alonso‐Betanzos · 2009
In this paper, a novel supervised architecture for binary classification based on local modelling and information theory is described. The architecture is composed of two steps: in the first one, a separating borderline between the two classes is piecewise constructed by a set of centroids calculated by a modified clustering algorithm, based on information theory; each of these centroids define a region where, in the second step of the proposed architecture, a hyperplane is constructed and adjusted by means of one-layer neural networks. This new method allows for binary classification while maintaining adequate use of computational resources, a common problem for machine learning methods. The proposed architecture is applied over classical benchmark classification problems and data sets, and its results are compared with those obtained by other well-known statistical and machine learning classifiers.