A Bayesian Framework for Online Classifier Ensemble
Qinxun Bai, Henry Lam, Stan Sclaroff · 2014
We propose a Bayesian framework for recur-sively estimating the classifier weights in online learning of a classifier ensemble. In contrast with past methods, such as stochastic gradient descent or online boosting, our framework estimates the weights in terms of evolving posterior distribu-tions. For a specified class of loss functions, we show that it is possible to formulate a suitably de-fined likelihood function and hence use the poste-rior distribution as an approximation to the global empirical loss minimizer. If the stream of train-ing data is sampled from a stationary process, we can also show that our framework admits a supe-rior rate of convergence to the expected loss min-imizer than is possible with standard stochastic gradient descent. In experiments with real-world datasets, our formulation often performs better than online boosting algorithms. 1.