Bivariate splits and consistent split criteria in dichotomous classification trees
David J. Lubinsky · 1994
We extend the recursive partitioning approach to classifier learning to use more complex splits at each decision node. To do this a new split criterion is derived. Efficient algorithms for finding optimal splits under the new criterion for optimal linear, rectangular, and corner splits are presented. These allow decision trees to model non-orthogonal structure and result in decision trees that often perform better in terms of reclassification accuracy than traditional methods, as well as being significantly smaller. Second, we discuss the lack of consistency of traditional split criteria such as entropy and Gini. These criteria do no optimize accuracy, whereas predictive accuracy is the most important metric by which trees are measured and is the only criterion which leads to consistent estimates of thresholds. A modification to the standard tree growing algorithm that ensures consistent trees is proposed and is shown to give smaller trees with better performance on a number of datasets. We show that traditional split criteria might miss the optimal consistent split by large amounts and under certain conditions can generate trees of unbounded size.