Bayesian criteria based on universal measures
Joe Suzuki · International Symposium on Information Theory and its Applications · 2012
In the minimum description length (MDL) and Bayesian criteria, we construct description length of data zn = z 1 ···z n of length n such that the length divided by n almost converges to its entropy rate as n → ∞, assuming z i is in a finite set A. In model selection, if we knew the true probability P of zn Є An, we would choose a model F such that the posterior probability of F given zn is maximized. But, in many situations, we use Q : An → [0,1] such that Σ z nЄAn Q(zn) ≤ 1 rather than P because only data zn are available. In this paper, we consider an extension such that each of the attributes in data can be either discrete or continuous. The main issue is what Q is qualified to be an alternative to P in the generalized situations. We propose the condition in terms of the Radon-Nikodym derivative of P with respect to Q, and give the procedure of constructing Q in the general setting. As a result, we obtain the MDL/Bayesian criteria in a general sense.