In-the-loop training algorithm for neural network implementation with digital weights
Jinming Yang, GRAHAM A. JULLIEN, Majid A. Ahmadi, W. C. Miller · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1998
In this paper, we propose a training algorithm for VLSI neural networks with digital weights and analog neurons using in-the-loop training strategy. The use of digital weights in a neural network implementation imposes new issues that are not present in simulation environments. One of the problems is that a neural network implementation will not work properly when using the digitized version of the continuous weight solution. This phenomenon is especially evident when the digital weight resolution is very low due to some fabrication constraints. In this paper the training strategies for dealing with digital weights are investigated. The proposed training algorithm is by measuring the sensitivity of each weight to its error function and then by perturbing the weights of higher sensitivity values to perform retraining process. Our experimental results indicate that the algorithm is feasible and particularly suitable for the digital weights with low number of bits.