A Study of Parallel Perturbative Gradient Descent
D. Lippe, J. Alspector · neural information processing systems · 1994
We have continued our study of a parallel perturbative learning method [Alspector et al., 1993] and implications for its implementation in analog VLSI. Our new results indicate that, in most cases, a single parallel perturbation (per pattern presentation) of the function parameters (weights in a neural network) is theoretically the best course. This is not true, however, for certain problems and may not generally be true when faced with issues of implementation such as limited precision. In these cases, multiple parallel perturbations may be best as indicated in our previous results.