The Pruning Algorithm of Parallel Shared Decision Tree Based on Hadoop
Xiangyang She, Tao Lv, Xiaojian Liu · 2017
Shared knowledge mining refers to learn the sharing knowledge of different things, and applying the learned knowledge to the unknowns to accelerate the recognition of them. For the problems of low efficiency and the traditional memory classification algorithm unable to deal with massive data, then combined with the cloud computing technology, A Parallel Shared Decision Tree (PSDT) algorithm had been proposed. Although this algorithm improved the efficiency, the performance still needs to be optimized for leaving the influence of the training set noise out of consideration. So, in this paper, based on the PSDT algorithm, a parallel shared decision tree imprecise error pruning (PSDT-IEP) algorithm is proposed. In our algorithm, we reduced the impact of unreliability by using the classification number of the uncertainty probability error of the data set to prune, which improves the accuracy of the algorithm. Along with the increase of the data set, the superiority of PSDT-IEP is more obvious.