A Novel Pessimistic Decision Tree Pruning Approach for Classification
Al Maruf Hassan, Abir Hossain Amee, Md Iftekhar Hossain, Sara Ferdous Khan, Dewan Md. Farid · 2023
Decision Tree (DT) is a top-down recursive divide-conquer machine learning algorithm which commonly used in many real-world classification tasks in supervised learning. The beauty of decision tree induction is its simplicity and non-linear relation among the features don’t affect the classification performance. Constructing a DT is straightforward and comprehensible. Most of the machine learning classifiers work as a black box model that we don’t understand what is happing inside the algorithm. Decision tree classifier gives us a set of classification rules which lucidly illustrate the classification process. Decision tree pruning is one of the methods that used to deal with overfitting problems in supervised learning. DT pruning methods are bifurcated into pre-pruning and post-pruning methods. Post-pruning methods are two/three times faster than pre-pruning methods. In this paper, we have proposed a novel idea to introduce pessimistic decision tree pruning method with pruning set for classification task. The proposed method finds a best prune tree among the set of prune trees with cost-complexity and recall values. The set of pruned trees are considered from several decision trees or random decision trees. The best pruned tree is selected which has small in size with highest classification accuracy. We evaluated the effectiveness of the proposed pruning approach using ten real benchmark datasets sourced from the repository of UCI machine learning. The experimental data shows us that the proposed pruning method performs better than existing tree pruning techniques.