On Inconsistent Bayes Estimates in the Discrete Case
David A. Freedman, Persi W. Diaconis · The Annals of Statistics · 1983
Consider sampling from an unknown probability distribution on the integers. With a tail-free prior, the posterior distribution is consistent. With a mixture of a tail-free prior and a point mass, however, the posterior may be inconsistent. This is likewise true for a countable mixture of tail-free priors. Similar results are given for Dirichlet priors.