Hardware Implementation of Hyperbolic Tangent Activation Function for Floating Point Formats
T.K.R Arvind, Marcel Brand, Christian J. A. Heidorn, Srinivas Boppu, Frank Hannig, Jürgen Teich · 2020
In this paper, we present the efficient hardware implementation of hyperbolic tangent activation function, which is most widely used in artificial neural networks for accelerating machine learning applications. The proposed design considers the floating point representation of numbers for the first time, the nonlinear nature of the activation function while sampling, and uses a lookup table for implementation. The unique way of dividing the input range into bins which follows the binary pattern reduces the hardware implementation cost. Furthermore, the input data itself is used as the address for lookup table; thus, no extra cost involved in hashing the lookup table and involves only one memory access time resulting in faster and efficient hardware implementation. Our design proves to be 3× faster when compared to similar hardware implementations using CMOS 90 nm process.