Nonparametric Bayes
David B. Dunson · Wiley StatsRef: Statistics Reference Online · 2017
Abstract Bayesian statistical methods consist of three key components: (i) a prior distribution characterizing uncertainty in parameters before observing current data; (ii) a likelihood function for the current data; and (iii) a loss function characterizing the price to be paid for errors in estimation, prediction, or decision‐making. Using Bayes' rule, the prior distribution is updated to include information in the likelihood function to obtain a posterior distribution quantifying uncertainty about the parameters. Parametric Bayesian models typically involve finitely many unknown parameters and require strong modeling assumptions. For example, one may assume that a response variable is normally distributed about a mean, which varies linearly with predictor variables. The posterior distribution, and associated predictions and decisions, will then be critically dependent on the accuracy of these modeling assumptions. Nonparametric Bayes (npB) methods seek to remove this limitation by defining more flexible models that can approximate any “true” data‐generating model in a large class. For example, instead of assuming that data are normally distributed, one may place a prior on the unknown density of the data. For this prior to be “nonparametric,” it must satisfy a large support property – meaning that the prior has positive probability of generating a density in an arbitrarily small neighborhood of any density in a large class. This article provides a brief overview of the npB philosophy, describing some basic details of canonical models, including Gaussian processes and Dirichlet process mixtures.