Contextual Word Representation and Deep Neural Networks-based Method for Arabic Question Classification
Hamza Alami, Noureddine En-Nahnahi, Saïd Ouatik El Alaoui · Advances in Science Technology and Engineering Systems Journal · 2020
Contextual continuous word representation showed promising performances in different natural language processing tasks.It stems from the fact that these word representations consider the context in which a word appears.But until recently, very little attention was paid to the contextual representations in Arabic question classification task.In the present study, we employed a contextual representation called Embeddings from Language Models (ELMo) to extract semantic and syntactic relations between words.Then, we build different deep neural models according to three types: Simple models, CNN and RNN mergers models, and Ensemble models.These models are trained on Arabic questions corpus to optimize the cross entropy loss given questions representations and their expected labels.The dataset consists of 3173 questions labeled according the Arabic taxonomy and an updated version of the Li & Roth taxonomy.We performed various comparisons with models based on the widely known contextfree word2vec word representation.These evaluations confirm that ELMo representation achieves top performances.The best model scores up to 94.17%, 94.07%, 94.17% in accuracy, macro F1 score, and weighted F1 score, respectively.