Examining the Relationship Between Majority Vote Accuracy and Diversity in Bagging and Boosting
Christopher J. Whitaker, Ludmila Ilieva Kuncheva, Bangor Bangor · 2003
Much current research is undertaken into combining classifiers to increase the classification accuracy. We show, by means of an enumerative example, how combining classifiers can lead to much greater or lesser accuracy than each individual classifier. Measures of diversity among the classifiers taken from the literature are shown to only exhibit a weak relationship with majority vote accuracy. Two commonly used methods of designing classifier ensembles, Bagging and Boosting, are examined on benchmark datasets. Bagging is shown to produce teams with little diversity or improvement in accuracy, while Boosting tends to produce more diverse classifier teams showing an improvement in accuracy.