Theoretical Views and Results
Ludmila Ilieva Kuncheva · 2004
Chapter 9 is a collection of theoretical results from the literature on combining classifiers. We consider the equivalence between some combination methods under specified conditions: minimum=maximum, majority vote=median. We reproduce the derivation of the added error for the mean combination rule as a function of the pairwise correlation between the classifiers in the ensemble. A similar derivation for the weighted average is also given and the optimal weights are found. The ensemble error for independent classifiers is formulated for normal and uniform distribution of the individual errors. The error formulas are given for the following combiners: minimum, maximum, mean, median, majority vote, and the oracle.