Maximum Likelihood
Derek A. Roff · Cambridge University Press eBooks · 2006
Introduction Suppose that we have a model with a single parameter, θ , that predicts the outcome of an event that has some numerical value y . Further, suppose we have two choices for the parameter value, say θ 1 and θ 2 , where θ 1 predicts that the numerical value of y will occur with a probability p 1 and θ 2 predicts that the numerical value of y w`ill occur with a probability p 2 . Which of the two choices of θ is the better estimate of the true value of θ ? It seems reasonable to suppose that the parameter value that gave the highest probability of actually observing what was observed would be the one that is also closer to the true value of θ . For example, if p 1 equals 0.9 and p 2 equals 0.1, then we would select θ 1 over θ 2 , because the model with θ 2 predicts that one is unlikely to observe y , whereas the model with θ 1 predicts that one is quite likely to observe y . We can extend this idea to many values of θ by writing our predictive model as a function of the parameter values, ϕ ( θ i ) = p i , where i designates particular values of θ . More generally, we can dispense with the subscript and write ϕ ( θ ) = p , thereby allowing θ to take on any value. By the principle of maximum likelihood we select the value of θ that has the highest associated probability, p .