A digital circuit design of hyperbolic tangent sigmoid function for neural networks
Che‐Wei Lin, Jeen-Shing Wang · 2008
This paper presents a digital circuit design approach for a commonly used activation function, hyperbolic tangent sigmoid functions, for neural networks. Our design concept for such a nonlinear function is to approximate the function of its first-order derivative by piece-wise linear functions first, then to obtain the estimate of the original function by integrating the approximated function of the first-order derivative by a digital circuit. The average error and maximum error of the proposed approximation approach are in the order of 10−3and 10−2, respectively in the software simulation. The hardware implementation of the proposed method consumes only one multiplication and one addition/subtraction ALU with the aid of resource sharing. The performance of our circuit has been validated by a neural network for a system identification problem in the software simulation.