Variance Penalizing AdaBoost
Pannagadatta K. Shivaswamy, Tony Jebara · 2011
This paper proposes a novel boosting algorithm called VadaBoost which is mo-tivated by recent empirical Bernstein bounds. VadaBoost iteratively minimizes a cost function that balances the sample mean and the sample variance of the expo-nential loss. Each step of the proposed algorithm minimizes the cost efficiently by providing weighted data to a weak learner rather than requiring a brute force evaluation of all possible weak learners. Thus, the proposed algorithm solves a key limitation of previous empirical Bernstein boosting methods which required brute force enumeration of all possible weak learners. Experimental results con-firm that the new algorithm achieves the performance improvements of EBBoost yet goes beyond decision stumps to handle any weak learner. Significant perfor-mance gains are obtained over AdaBoost for arbitrary weak learners including decision trees (CART). 1