Investigating Fast BiLSTM Neural Networks for Arabic Language Applications
Abrar K. Samara, Gheith A. Abandah · 2021
Deep neural networks are providing efficient solutions for many natural language processing (NLP) applications. Recently, recurrent neural networks based on the long short-term memory (LSTM) cells are successfully used in various solutions for the Arabic language. However, these networks are complex and require long training times on large datasets. In this paper, we present an investigation for fast alternatives to the commonly used bidirectional LSTM (BiLSTM) layers. As case studies, we evaluate alternative networks for two Arabic applications: classifying poems and diacritizing text. We concentrate in this paper on networks based on the CUDA deep neural network (CuDNNLSTM) layer for its many advantages. We present thorough performance evaluation of BiLSTM and BiCuDNNLSTM networks including model complexity, training time, and accuracy for both applications. We recommend BiCuDNNLSTM networks as alternative for BiLSTM networks because they improve training time by 65 and 21 times for poem classification and diacritization, respectively. Moreover, the BiCuDNNLSTM networks provide similar or higher accuracies (99.16% vs. 98.41 % for diacritization). The advantage of this recommend layer originates from its efficient utilization of the GPU.