On heuristics for learning model trees
Celine Vens, Hendrik Blockeel · Lirias · 2003
Induction of decision trees is a popular learning technique, not just for classi cation but also for numerical prediction (regression). The term \\model trees" is commonly used for trees that in their leaves contain some (usually linear) regression model. Popular implementations of model tree learners use reduction of variance as a heuristic for selecting tests during the tree construction process. In this paper, we show that systems employing this heuristic may exhibit pathological behaviour in some quite simple cases. This is not visible in the predictive accuracy of the tree, but it reduces its explanatory power. We propose an alternative heuristic that yields equally accurate but simpler trees with better explanatory power, and this at little or no additional computational cost. 1