Stochastic approximation algorithms for statistical estimation
Shaolin Li · Library and Archives Canada (Government of Canada) · 1996
This thesis presents some broadly applicable algorithms for computing maximum likelihood estimates (MLE) from the incomplete data based on the stochastic approximation (SA) proposed by Robbins and Monro (1951). The usual approach for such problems is the EM algorithm. In many interesting examples, however, it is impossible to carry out either the E-step or the M-step of the EM algorithm. Although some remedial EM algorithms were developed, these algorithms could be very expensive numerically, especially when both the E-step and the M-step become intractable. The SA algorithms proposed are appealing because they avoid computing expectation within iterations and are easy to implement. These advantages are reinforced by a discussion of some examples illustrating how these SA algorithms can succeed while the EM algorithm is intractable. Theory showing convergence of these algorithms along with the rate of their optimal convergence is developed. Moreover, the thesis also explores some theoretical issues about the robustness, consistency and asymptotic theory of the SA estimation of MLE with incomplete data in the double array sense.