Bayesian Inference and the Optimality of Maximum Likelihood Estimation
James J. Higgins · International Statistical Review · 1977
Bayesian methods have been widely discussed from both a philosophical and a practical point of view. Lindley's review (1970) contains interesting arguments in support of Bayesian methods, and his monograph contains an extensive bibliography of the subject area. Although accepted by many, Bayesian methods have also been severely criticized, being among the critics. Savage (1975) reviewed Fisher's work, and his paper provides insights into Fisher's objections to Bayesian methods. An extensive bibliography of the Bayesian subject area is also contained in Savage's paper. From Savage's comments [1975, pp. 456-57], one learns that regarded the maximum likelihood estimate (MLE) as being the most informative estimate of an unknown parameter. Certainly, this view has theoretical support in an asymptotic sense, but as Efron (1975) stated, Fisher believes that the MLE is optimum as an information gathering statistic in finite samples, not just asymptotically. The papers of Godambe (1960) and Bhopkar (1972) tend to support Fisher's view of the MLE. Here we present some proved arguments which we believe also support Fisher's view. Ironically, our arguments utilize the Bayesian notions of prior and posterior distributions. It should be noted that we do not attempt to define a mathematical measure of information. Rather, we regard information simply as acquired knowledge. Suppose we assign a prior probability density function g (0) to a parameter 0, and suppose we assume that a vector of observations X has a sampling density f(x i 8). The prior density expresses our knowledge of the value of 0 before experimentation, and the posterior density defined by