On the Averaged Stochastic Approximation for Linear Regression
László Györfi, Harro Walk · SIAM Journal on Control and Optimization · 1996
For a linear regression function the average of stochastic approximation with constant gain is considered. In case of ergodic observations almost sure convergence is proved, where the limit is biased with small bias for small gain. For independent and identically distributed observations and also under martingale and mixing assumptions, asymptotic normality with $(n^{{{ - 1} / 2}} )$-convergence order is obtained. In the martingale case the asymptotic covariance matrix is close to the optimum one if the gain is small.