Multi-modal sentiment analysis based on graph neural network
Boao Li · Applied and Computational Engineering · 2023
Thanks to popularity of social media, people are witnessing the rapid proliferation of posts with various modalities. It is worth noting that these multi-modal expressions share certain characteristics, including the interdependence of objects in the posted images, which is sometimes overlooked in previous researches as they focused on single image-text posts and pay little attention on obtaining the global features. In this paper, a neural network with multiple channels for image-text sentiment detection is proposed. The first step is to encode text and images to capture implicit tendencies. Then the introduction of this model obtains multi-modal expressions by collecting the shared characteristics of the dataset. Finally, the attention mechanism provides reliable predictions of the sentiment tendencies of the given pairs of image-text data. The results of experiments conducted on two publicly available datasets crawled from Twitter prove the reliability of the model on multi-modal sentiment detection, since the model precedes previously proposed models in the main evaluating criteria.