DynaBoost: Combining Boosted Hypotheses in a Dynamic Way
Perry D. Moerland, Eddy Mayoraz · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · 1999
We present an extension of Freund and Schapire's AdaBoost algorithm that allows an input-dependent combination of the base hypotheses. A separate weak learner is used for determining the input-dependent weights of each hypothesis. The error function minimized by these additional weak learners is a margin cost function that has also been shown to be minimized by AdaBoost. The weak learners used for dynamically combining the base hypotheses are simple perceptrons. We compare our dynamic combination model with AdaBoost on a range of binary and multi-class classication problems. It is shown that the dynamic approach signicantly improves the results on most data sets when (rather weak) perceptron base hypotheses are used, while the dierence in performance is small when the base hypotheses are MLPs.