Randomized approximate aggregating strategies and their applications to prediction and discrimination
Kenji Yamanishi · 1995
The effectiveness of Bayes methods and their generalization called the aggregating strategy has been demonstrated in statistics, game theor y, learning theory, etc.They often, however, suffer from computational difficulties in implementation since the exact numerical values of the Bayes posterior probabilities and the Bayes post erior means (or corresponding quantities in the aggregating strategy) are not always analytically or computationally tractab~e and must be approximately computed.This paper introduces methods of efficient approximation of Bayes methods (aggregating strategy) and demonstrates their effectiveness in the on-line prediction and discrimination scenarios.The algorithms introduced use randomizing techniques based on the Markov chain Monte Carlo method, which has extensively been explored in the context of computational statistics and statistical mechanics.We give a rigorous analysis with regard to 1) how well the algorithms can approximate true Bayes methods (aggregating strategy) and 2) how efficiently they work, both for the prediction and discrimina-tio~scenarios.The trade-off between the issues 1) and 2) is analyzed through the number of random samplings that the algorithms make.