Statistical Learning Theory of the LMS Algorithm Under Slowly Varying Conditions, Using the Langevin Equation
Simon Haykin · 2006
The paper begins with a brief description of the Langevin equation of nonequilibrium thermodynamics. In so doing, I set the stage for analyzing the statistical learning behavior of the standard LMS algorithm, operating under the assumption of a small step-size parameter. In particular, it is shown that by making three justifiable assumptions and then applying the unitary similarity transformation, the transformed formulation of the LMS algorithm takes on the discrete-time version of a Langevin equation for each natural mode of the algorithm. Experimental results are presented, which support practical validity of the LMS learning theory.