Log AdaBoost: Optimizing Polylog loss function to improve the generalization performance of AdaBoost
Meijin Lin, Haoyuan Luo · 2022 37th Youth Academic Annual Conference of Chinese Association of Automation (YAC) · 2022
As an ensemble learning algorithm, the classic AdaBoost has achieved incredible success in both classification and regression problems, but its generalization ability is still unsatisfactory. A modified version of the Real AdaBoost which is called Gentle AdaBoost was proposed by Friedman. Gentle AdaBoost which is more robust than Real AdaBoost optimizes the exponential loss function by using the Newton-Raphson stepping. In this paper, a more moderate variant of AdaBoost, called Log AdaBoost, is proposed for training classification model. Firstly, to be more tolerant for outliers, we optimized polylog loss function instead of the exponential loss function by taking gradient descent method. Moreover, a new weight updating strategy is taken to find the weak classifier most relevant to the label under the current weight distribution. In other word, the weight is updated towards the negative gradient of the function to select the optimal weak classifier. The experimental results show that compared with Real AdaBoost and Gentle AdaBoost, the proposed method has better performance in generalization ability.