Tree-Based Ensemble Models, Algorithms and Performance Measures for Classification
John Tsiligaridis · Advances in Science Technology and Engineering Systems Journal · 2023
An ensemble method is a Machine Learning (ML) algorithm that aggregates the predictions of multiple estimators or models.The purpose of an ensemble module is to provide better predictive performance than any single contributing model.This can be achieved by producing a predictive model with reduced variance using bagging, and bias using boosting.The Tree-Based Ensemble Models with Decision Tree (DT) as base model is the most frequently used.On the other hand, there are some individual Machine Learning algorithms that can provide more competitive predictive power to the ensemble models.It is a problem, and this issue is addressed here.This work has two parts.The first one presents a Projective Decision Tree (PA) based on purity measure.Next node criterion (CNN) is also used for node decision making.In the second part, two sets of algorithms for predictive performance are presented.The Tree-Based Ensemble model includes bagging and boosting for homogeneous learners and a set of known individual algorithms.Comparison of two sets is performed for accuracy.Furthermore, the changes of bagging and boosting ensemble performance under various hyperparameters are also investigated.The datasets used are the sonar and the Breast Cancer Wisconsin (BCWD) from UCI site.Promising results of the proposed models are accomplished.