A Comparison of Boosting Performance by Learner
The Korean Data Analysis Society, Ho-Jung Kim, HyungJun Cho · The Korean Data Analysis Society · 2024
Ensemble learning is a learning method that combines several models to create a model with strong performance, showing excellent performance in many fields of real life. Ensemble learning methods largely have boosting and bagging techniques, and the difference between these two methods arises from whether the learner is trained sequentially or independently. In most boosting methodologies, decision trees are used as learners to perform well, and there are papers comparing them. However, models other than decision trees can also be used as learners of boosting, thus in this paper, various learning methods were applied and the results were compared. Limited to the situation of binomial classification problems, the accuracy of boosting was compared by applying a generalized linear model, a support vector machine, a K-nearest neighbors, discriminant analysis, and an artificial neural network as learners. As a result, out of all six learning methodologies, boosting based on decision trees was the best, and boosting based on generalized linear models, K-nearest neighbors, and artificial neural networks generally showed good performance, and worst performance based on support vector machines and discriminant analysis.