From Random Forest to an interpretable decision tree - An evolutionary approach
Krzysztof Jurczuk, Marcin Czajkowski, Marek Krętowski · 2023
Random Forest (RF) is one of the most popular and effective machine learning algorithms. It is known for its superior predictive performance, versatility, and stability, among other things. However, an ensemble of decision trees (DTs) represents a black-box classifier. On the other hand, interpretability and explainability are ones of the top artificial intelligence trends, to make predictors more trustworthy and reliable. In this paper, we propose an evolutionary algorithm to extract a single DT that mimics the original RF model in terms of predictive power. The initial population is composed of trees from RF. During evolution, the genetic operators modify individuals (DTs) and exploit the initial (genetic) material. e.g., splits/tests in the tree nodes or more expanded parts of the DTs. The results show that the classification accuracy of a single DT predictor is not worse than that of the original RF. At the same time, and probably most importantly, the resulting classifier is a single smaller-size DT that is almost self-explainable.