Integer weight training by differential evolution algorithms
Vassilis P. Plagianakos, Dimitris G. Sotiropoulos, Michael N. Vrahatis · 1998
Abstract In this work differential evolution strategies areapplied in neural networks with integer weightstraining. These strategies have been intro-duced by Storn and Price [Journal of GlobalOptimization, 11, pp. 341–359, 1997]. Inte-ger weight neural networks are better suited forhardwareimplementation as compared with theirreal weight analogous. Our intention is to givea broad picture of the behaviour of this class ofevolution algorithms in this difficult task. Sim-ulation results show that this is a promising ap-proach. 1 Introduction An artificial Feedforward Neural Network (FNN)consists of many interconnected identical simpleprocessing units, called neurons. Each neuroncalculates the dot productof the incoming signalswith its weights, adds the bias to the resultant,and passes the calculated sum through the acti-vation function. In a multilayer feedforward net-work the neurons are organized into layers withno feedback connections.FNNs can be simulated in software, but in or-der to be utilized in real life applications, wherefast speed of execution is required, hardware im-plementation is needed. The natural implemen-tation of an FNN – because of its modularity – isa parallel one. The problem is that the conven-tional multilayer FNNs, which have continuousweights is expensive to implement in digital hard-ware. Another major implementation obstacle isthe weight storage. FNNs having integer weightsand biases are easier and less expensive to imple-ment in electronics as well as in optics and thestorage of the integer weights is much easier toachieved.A typical FNN consisting of L layers, wherethe first layer denotes the input, the last one, L,is the output, and the intermediate layers are thehidden layers. It is assumed that the (l-1) layerhas N