Lower bounds for Boosting with Hadamard Matrices
Manfred K. Warmuth, Xinhua Zhang · 2013
Boosting algorithms can be viewed as a zero-sum game. At each iteration a new column / hypothesis is chosen from a game matrix representing the entire hypotheses class. There are algorithms for which the gap between the value of the sub-matrix (the t columns chosen so far) and the value of the entire game matrix is O( q logn t ). A matching lower bound has been shown for random game matrices for t up to n where 2 (0; 1 ). We conjecture that with Hadamard matrices we can build a certain game matrix for which the game value grows at the slowest possible rate for t up to a fraction of n.