Comment: Bayes, Oracle Bayes and Empirical Bayes
Aad van der Vaart · Statistical Science · 2019
Empirical Bayes methods are intriguing, and have gained in significance by present day big data applications.Despite their early introduction, they are still not fully understood.It is a pleasure to read the review by a "statistical rock star" [13], who stood at the beginning of the methods and more recently opened our eyes to their importance for large scale inference.Empirical Bayes combines Bayesian ways of thinking about data and what some call "frequentist" methods, often maximum likelihood.The main point of my discussion is to highlight connections to nonparametric and high-dimensional Bayesian methods, which have seen a big development in the past 20 years.In the second paragraph of Section 6, Efron writes: "which is to say that standard Bayes is finite Bayes with N = ∞" and goes on to describe a fully Bayesian approach (consisting of a hyperprior h(g) on the density of the parameters θ i ) as an "uncertain task".I may not be full Bayes enough to say this with absolute certainty, but would think that nowadays most Bayesians would politely disagree and consider the setting a standard one, with a Dirichlet process prior as a "default" choice [17,18,1,20].Then the setting is described by the hierarchy:This is the model of Efron's Sections 1-4 augmented with a prior on G, and could still be preceded by extra levels to construct the parameter α (a finite distribution) of the Dirichlet process DP(α), in particular its total mass (called "prior precision").We restrict to the case that the observations in step three are Gaussian; it would be worth while to extend our discussion to Poisson observations, as in Efron's Section 5.