Design Smart NNTrees Based on the -rule
Qiangfu Zhao · 2005
Neural network tree (NNTree) is a hybrid learning model with the overall structure being a decision tree (DT), and each non-terminal node containing an expert neural network (ENN). Generally speaking, NNTrees outperform conventional DTs because more complex and possibly better features can be extracted by the ENNs. So far we have studied several genetic algorithms (GAs) for designing the NNTrees. These algorithms are computationally expensive, and the NNTrees obtained are often very large. In this paper, we propose a new approach based on the -rule, which is a non-genetic evolutionary algorithm proposed by the author several years ago. The key point is to propose a heuristic method for defining the teacher signals for the examples assigned to a non-terminal node. Once the teacher signals are defined, the ENNs can be trained quickly using the -rule. Experiments with several public databases show that the new approach can produce smart NNTrees quickly and effectively. I. INTRODUCTION to propose a heuristic method for defining the teacher signals for the examples assigned to a node. Once the teacher signals are defined, the ENNs can be trained quickly using the - rule. Note that the ENNs used in this paper are actually the nearest neighbor based multilayer perceptrons (NN-MLPs). If we use the conventional multilayer perceptrons (MLPs), the well-known back propagation (BP) algorithm can also be used after defining the teacher signals. We choose NN-MLPs here because the structure (the number of hidden neurons) of the network can be determined automatically using the -rule. Experiments with several public databases show that the new approach can produce smart NNTrees quickly and effectively. By smart here we mean that an NNTree is small and generalize well.