Bayesian Estimation of Mixing Distributions
John E. Rolph · The Annals of Mathematical Statistics · 1968
Let $\\mathscr{Q} = \\{Q(t): 0 \\leqq t \\leqq 1\\}$ be a family of probability distributions on the positive integers parameterized on [0, 1], that is \\begin{equation*}\\tag{1} P_t(X = x) = q_x(t);\\quad x = 1,2,\\cdots.\\end{equation*} If $G$ is a distribution on $\\lbrack 0, 1\\rbrack$ the distribution of $X$ is a $G$-mixture over $\\mathscr{Q}$ if \\begin{equation*}\\tag{2} P_G(X = x) = \\int^1_0 q_x(t) dG(t) = q_x(G).\\end{equation*} $G$ is called the mixing distribution. It is assumed at the outset that the family $\\mathscr{Q}$ is known to be identifiable, that is if $q_x(G_1) = q_x(G_2)$ for $x = 1,2,\\cdots$, then $G_1 = G_2$. See [12] and [13] for conditions insuring identifiability. Thus it makes sense to attempt to estimate $G$ when one has independent observations on $X$. Some work on estimating $G$ has been done when the mixture is finite [4], [2], [9] and for special $\\mathscr{Q}$'s [6], [14]. The problem is of interest not only in an estimation context, but also in the construction of empirical Bayes decision procedures [9]. Our approach is to define a prior distribution on possible values of $G$ and then construct consistent Bayes estimates of $G$ from the posterior distribution. Section 2 gives the needed background on moment spaces, sets up the prior distribution and derives the posterior distribution. In Section 3, the Bayes estimates are defined while Section 4 proves the consistency of the posterior distribution and thus of the estimates. Here, Theorem 1 is not directly applicable to our problem, but is included because of its possible independent interest. Sections 5 and 6 generalize the earlier results.