$\log$-Sigmoid Activation-Based Long Short-Term Memory for Time-Series Data Classification
Priyesh Ranjan, Pritam Khan, Sudhir Kumar, Sajal Kumar Das · IEEE Transactions on Artificial Intelligence · 2023
With the enhanced usage of artificial-intelligence-driven applications, the researchers often face challenges in improving the accuracy of data classification models, while trading off the complexity. In this article, we address the classification of time-series data using the long short-term memory (LSTM) network while focusing on the activation functions. While the existing activation functions, such as sigmoid and$\tanh$, are used as LSTM internal activations, the customizability of these activations stays limited. This motivates us to propose a new family of activation functions, called$\log$-sigmoid, inside the LSTM cell for time-series data classification and analyze its properties. We also present the use of a linear transformation (e.g.,$\log \tanh$) of the proposed$\log$-sigmoid activation as a replacement of the traditional$\tanh$function in the LSTM cell. Both the cell activation and recurrent activation functions inside the LSTM cell are modified with$\log$-sigmoid activation family while tuning the$\log$bases. Furthermore, we report a comparative performance analysis of the LSTM model using the proposed and the state-of-the-art activation functions on multiple public time-series databases.