Approximation of the sigmoid function and its derivative using a minimax approach
Jason Schlessman · 2002
Accurate function approximation is ~ssential for a number of applications, including signal processing, multimedia, and neural networks. The sigmoid function and its derivative are particularly important to neural network applications. Typically, the sigmoid function is used as a learning function and a threshold determinant for training neural networks. The sigmoid's derivative is used as an activation function for neural networks. As demands on neural network speed and accuracy increase, methods for effective digital approximation of the sigmoid function is increasingly important. It is with this in mind that an overview of previously employed design choices for sigmoid apprximation are presented, as well as a discussion of the mathematical properties of the sigmoid function and its derivative. In addition, a novel minimax approach to approximating the sigmoid and its derivative is presented. This approach was developed with reduction of maximum error and simplicity of approximation as improtant design criteria. Designs for the sigmoid and its derivative are shown for seven to eleven fractional bits of accuracy. The designs were implemented in VHDL and synthesized to an FPGA device and an ASIC library. For both technologies, the designs increase consistently in both area and delay with in('l'(\ased numbers of accurate bits.