Close Approximations of Sigmoid Functions by Sum of Steps for VLSI Implementation of Neural Networks
Valeriu C. Beiu, Jan A. Peperstraete, Joos P. L. Vandewalle, Rudy Lauwereins · 1994
This paper is devoted to show that there are simple and accurate ways to compute a sigmoid nonlinearity and its derivative in digital hardware by sum of steps, and that threshold gate implementation of such algorithms are area-efficient when compared to other known methods. 1. OVERVIEW The paper starts by describing classical solutions for digital hardware implementation of the nonlinear activation functions used by artificial neurons, the accent falling on sigmoid nonlinearities. Fresh results from the known literature are mentioned and shortly compared (2. Classical Solutions). But even if approximation techniques are used, the computations involved are quite complex. That is why we introduce a particular sigmoid function (3. A Particular Sigmoid Function). It is not very difficult to show that this particular sigmoid function is equivalent with the classical sigmoid function if the amplification factor (gain) is changed by a constant. As this constant can be used to m...