Constructive Hybrid Decision Tree
Junhu Zhou · Chinese Journal of Computers · 2001
The work described in this paper was motivated by two factors. On one hand, symbolism and connectionism are two main streams of current machine learning technology. Since both approaches have their own weaknesses, it is believed that combining symbolism and connectionism may generate powerful learning schemes. On the other hand, most inductive learning approaches at present are selective ones that may generate poor results when the original attributes do not well describe the problems to be solved. It is believed that constructive induction that automatically constructs internal attributes or concepts in the learning process is helpful to circumvent such an obstacle. In this paper, a constructive hybrid decision tree learning algorithm named CHDT is proposed. CHDT utilizes symbolic learning to perform qualitative analysis and utilizes neural learning to perform following quantitative analysis, which simulates human reasoning process in some sense. CHDT tries to process the instances with pure symbolic decision tree in an instance space defined by only category attributes. Only when the instances cannot be processed, it resorts to neural networks in an instance space defined by continuous attributes. The neural learning algorithm employed by CHDT is FTART2, which is a field theory based adaptive resonance theory model. CHDT virtually embeds neural networks in the leaves of the decision tree. That is, CHDT marks the neural nodes in the learning process and trains only one neural network for all the marked nodes. CHDT employs a unique constructive induction mechanism. It uses a binary logic OR operator to construct new attributes through observing the topology of the trained decision tree. The construction process is repeatedly executed until the accuracy of a new tree is not better than an old tree. Experiments on three UCI data sets and a real-world data set show that CHDT can generate accurate and concise hybrid decision trees, where the accuracy profits from hybrid learning while the conciseness profits from constructive induction.