Bayes code with singular prior
Hayato Takahashi · Systems and Computers in Japan · 2003
Abstract We study the asymptotic code lengths of sequences generated by smooth parametric models. We study especially the cases such that the parameters are in a set of Lebesgue measure 0. In general, for smooth parametric models, it is known that, under a suitable condition, the lower bound of the code length is −logP̂ + (k/2)logn + o(logn), except for parameters in a set of Lebesgue measure 0, where P̂ is the maximum likelihood, k is the dimension of the model, and n is a sample size. It is known that MDL coding and Bayes coding achieve this lower bound. In this paper we construct a code by Bayes mixture with a singular prior. The code shown in this paper has a shorter code length than MDL coding (log coefficient less than k/2) when the parameters are in a set of measure 0, and is considered to be intermediate between Shannon coding and MDL coding. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(8): 22–32, 2003; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/scj.10358