CUET-NLP@DravidianLangTech-ACL2022: Investigating Deep Learning Techniques to Detect Multimodal Troll Memes
Md. Rakibul Hasan, Nusratul Jannat, Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque · 2022
With the substantial rise of internet usage, social media has become a powerful communication medium to convey information, opinions, and feelings on various issues.Recently, memes have become a popular way of sharing information on social media.Usually, memes is visuals with text incorporated into them and quickly disseminate hatred and offensive content.Detecting or classifying memes are challenging due to their region-specific interpretation and multimodal nature.This work presents a meme classification technique in Tamil developed by the CUET NLP team under the shared task (DravidianLangTech-ACL2022).Several computational models have been investigated to perform the classification task.This work also explored visual and textual features using VGG16, ResNet50, VGG19, CNN and CNN+LSTM models.Multimodal features are extracted by combining image (VGG16) and text (CNN, LSTM+CNN) characteristics.Results demonstrate that the textual strategy with CNN+LSTM achieved the highest weighted f 1 -score (0.52) and recall (0.57).Moreover, the CNN-Text+VGG16 outperformed the other models concerning the multimodal memes detection by achieving the highest f 1 -score of 0.49, but the LSTM+CNN model allowed the team to achieve 4 th place in the shared task.