Asymptotically minimax regret by Bayes mixtures for non-exponential families
Jun’ichi Takeuchi, Andrew R. Barron · 2013
We study the problems of data compression, gambling and prediction of a sequence xn= x1x2...xnfrom an alphabet X, in terms of regret with respect to various families of probability distributions. It is known that the regret of the Bayes mixture with respect to a general exponential families asymptotically achieves the minimax value when variants of Jeffreys prior are used, under the condition that the maximum likelihood estimate is in the interior of the parameter space. We discuss a modification of Jeffreys prior which has measure outside the given family of densities, to achieve minimax regret with respect to non-exponential type families, e.g. curved exponential families and mixture families. These results also provide characterization of Rissanen's stochastic complexity for those classes.