Construction of an Off-Centered Entropy for the Supervised Learning of Imbalanced Classes: Some First Results
Philippe Lenca, Stéphane Lallich, Benoît Vaillant · Communication in Statistics- Theory and Methods · 2010
In supervised learning, especially in decision tree induction, many measures are based on the concept of entropy. A major characteristic of entropies is that they take their maximal value when the distribution of the modalities of the class variable is uniform. In particular, to deal with the case when the a priori frequencies of the class variable modalities are imbalanced, we propose an off-centered entropy which takes its maximum value for a distribution fixed by the user. This distribution can be the overall distribution of the class variable or a distribution taking into account the costs of misclassification. Other authors proposed an asymmetric entropy. In this article, we propose an adaptive strategy of induction of decision trees based on these non centered entropies. The first experiments on ten data bases using the C4.5 induction tree show that both non centered entropies are promising in comparison with the classical Shannon entropy.