Training Neural Networks with 3–bit Integer Weights
Vassilis P. Plagianakos · 2010
In this work we present neural network train-ing algorithms, which are based on the differ-ential evolution (DE) strategies introduced by Storn and Price [Journal of Global Op-timization. 11:341–359, 1997]. These strate-gies are applied to train neural networks with 3–bit integer weights. Integer weight neu-ral networks are better suited for hardware implementation than their real weight anal-ogous. Moreover, we constrain the weights and biases in the range [−3, 3], thus, they can be represented by just 3 bits. This prop-erty reduces the amount of memory required and simplifies the digital multiplication oper-ation. Our intention is to present a broad picture of the behavior of this class of evolution al-gorithms in this difficult task. Simulation results from classical benchmarks show that these methods are promising, fast, and reli-able. 1