VLSI implementations of neural networks
Paul W. Hollis · PhDT · 1992
The purpose of this research is to explore various methods of implementing neural network architectures in hardware, with emphasis on learning networks and their associated algorithms. The approach taken utilizes hybrid (analog and digital circuitry) architectures which can benefit from the speed of parallel analog hardware, and from the ease of communication and control available with digital hardware. There are several areas of research which are highlighted in this manuscript. Three neuron models are described which have been characterized through simulation. Two of the models utilize traditional CMOS technology, while the third uses a newer BiCMOS technology. Test results are presented for hardware implementations of two of the models. The implications of the nonidealities inherent in these neuron circuits are examined, and methods of minimizing their effects are presented. It is demonstrated that the well-known backpropagation algorithm for training networks can be derived using an alternate neuron model to compensate for some of these nonidealities. The effects of precision constraints in hardware implementations are examined in the context of a hybrid architecture, and minimum precision requirements for effective learning are established for certain kinds of problems. A learning algorithm, designed to perform optimally when implemented in hardware, is introduced and verified with simulation data. This algorithm uses a perturbation method to implement gradient-descent learning, and to allow much simpler hardware than would be required by an implementation of standard backpropagation. A dynamic gain-adaption algorithm is used to maximize the effective resolution of the bounded weights which accompany any physical implementation. The use of the infinity-norm error measure facilitates the measurement of network error with simple hardware. Evidence is presented that network error information with a wide dynamic range can be coarsely encoded for weight updates without significantly affecting the quality of learning. This encoding method allows digital weight update calculations to be performed without using multipliers. Simulations of the full algorithm show that the quality and characteristics of learning are comparable between this algorithm and the standard backpropagation algorithm.