Bert-BiLSTM Model for Sentiment Analysis Using Contextual Embeddings and Bidirectional Dependencies
Jiajie Du, Haixing Zhao, Zhonglin Ye, Mingyuan Li · 2024
Sentiment analysis is crucial for understanding emotional tendencies in textual data. To address issues such as the inability of traditional machine learning models to effectively capture contextual semantic information and the sparsity of word vectors, this paper introduces a Bert-BiLSTM-based model. Two major segments form this model. The first part involves embedding through the Bert model, which can avoid the ambiguity caused by tokenization by using the semantic attributes of contextual word embeddings. The second part utilizes BiLSTM to capture bidirectional semantic dependencies within the context, consequently improving the effectiveness of the model. This model performs sentiment analysis on review content by first generating word clouds for positive and negative sentiments. It then analyzes high-frequency words and potential topics in the reviews by creating word clouds and constructing an LDA topic model. Experiments on public datasets demonstrate that, the Bert-BiLSTM model achieves an accuracy of 96.4%, compared to traditional methods, the Bert-BiLSTM model exhibits superior performance, proving its excellence in extracting the semantics of reviews over traditional methods.