Comparison of tracking algorithms for single layer threshold networks in the presence of random drift
Anthony Kuh · IEEE Transactions on Signal Processing · 1997
This paper analyzes the behavior of a variety of tracking algorithms for single-layer threshold networks in the presence of random drift. We use a system identification model to model a target network where weights slowly change and a tracking network. Tracking algorithms are divided into conservative and nonconservative algorithms. For a random drift rate of /spl gamma/, we find upper bounds for the generalization error of conservative algorithms that are /spl Oscr/(/spl gamma//sup 2/3/) and for nonconservative algorithms that are /spl Oscr/(/spl gamma/). Bounds are found for the perceptron tracker and the least mean square (LMS) tracker. Simulations show the validity of these bounds and show that the bounds are tight when /spl gamma/ is small and the number of inputs n is large. These results show that the perceptron tracker and the LMS tracker can work well in slowly changing nonstationary environments.