Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis
Amr El-Desoky Mousa, Björn Wolfgang Schuller · 2017
Traditional learning-based approaches to sentiment analysis of written text use the concept of bag-of-words or bag-of-ngrams, where a document is viewed as a set of terms or short combinations of terms disregarding grammar rules or word order.Novel approaches de-emphasize this concept and view the problem as a sequence classification problem.In this context, recurrent neural networks (RNNs) have achieved significant success.The idea is to use RNNs as discriminative binary classifiers to predict a positive or negative sentiment label at every word position then perform a type of pooling to get a sentence-level polarity.Here, we investigate a novel generative approach in which a separate probability distribution is estimated for every sentiment using language models (LMs) based on long short-term memory (LSTM) RNNs.We introduce a novel type of LM using a modified version of bidirectional LSTM (BLSTM) called contextual BLSTM (cBLSTM), where the probability of a word is estimated based on its full left and right contexts.Our approach is compared with a BLSTM binary classifier.Significant improvements are observed in classifying the IMDB movie review dataset.Further improvements are achieved via model combination.