Target-level Sentiment Analysis Based on Image and Text Fusion
Menglin Lu, Tongzhou Zhao, Chengbo Mao, Huibo Wang · 2022
Target-level sentiment analysis of social media posts has recently attracted increasing attentions, which task is to predict the sentimental label of target in posts published. Previous target-level sentiment analysis tasks usually only involve text modalities and fail to consider data sources of other modalities, which leads to low accuracy of sentiment analysis tasks. In order to improve the accuracy of sentiment analysis tasks, we propose a new target-level multimodal sentiment model that fuses features of visual and textual. Specifically, first, we use BERT and Bi-LSTM to extract the features of the target words and text, respectively, and then fuse them through a text fusion layer to obtain the final text representation. Next, we use ResNet-152 to extract image features, and then use attention mechanism to match target words and image features to extract target-related visual representations in the image. Finally, we linearly fuse the textual and visual representations using Multimodal fusion layer, and perform the sentiment label classification task after fusion.