Research on multi-modal hateful meme detection
Wanbo Li, Suying Liu · 2021
Research shows that the increase of social media usage is directly proportional to mental health. With the popularity of social networks, people increasingly like to vent their negative emotions on social platforms. These negative emotions, such as anxiety, tension, anger, depression, sadness and pain, have seriously affected our rational thinking. Long term impact of these negative emotions will affect the smooth progress of people's work and life, and even affect people's physical and mental health. At present, technology can detect negative comments that may have an impact on society. However, according to investigations and studies, social platforms are flooded with malicious pictures and texts composed of "harmless sentences" and "harmless images" to vent their negative emotions. The detection of these malicious pictures and text messages can help social media to correctly identify the expression of negative emotions and deal with them in a timely manner, thereby forming a good social atmosphere. The effective mode of current technology is for a single text or picture, so it is difficult and troublesome to solve this problem. Only text or image as a model can work well, but the detection of the meaning expressed by the combination of text and image cannot give appropriate results. This multimodality and benign confounding factors make it more difficult to detect malignant memes.