Universal Bayesian measures
Joe Suzuki · 2013
In the minimum description length (MDL) and Bayesian criteria, we construct description length of data zn= z1... znof length n such that the length divided by n almost converges to its entropy rate as n → ∞, assuming Ziis in a finite set A. In model selection, if we knew the true conditional probability P(zn|F) of zn∈ Angiven each F, we would choose F such that the posterior probability P(F|zn) of F given z" is maximized. But, in many situations, we use Q : An→ [0,1] such that ΣznϵAnQ(zn|F) ≤ 1 rather than P because only data znare available. In this paper, we consider an extension such that each of the attributes in data can be either discrete or continuous. The main issue is what Q is qualified to be an alternative to P in the generalized situations. We propose the condition in terms of the Radon-Nikodym derivative of P with respect to Q, and give the procedure of constructing Q in the general setting. As a result, we obtain the MDL/Bayesian criteria in a general sense.