Multimodal Sentiment Analysis in Text, Images, and GIFs Using Deep Learning
Deepak Minhas · BENTHAM SCIENCE PUBLISHERS eBooks · 2025
More and more people are inclined to use GIFs, videos, and photographs on social media as a way to convey their feelings and thoughts. We developed a Pythonbased multimodal sentiment analysis tool for various Twitter formats, taking into account not just the text of a tweet but also its accompanying GIFs and pictures, for more precise sentiment scoring. We employ fine-tuned CNN for image sentiment analysis, VADER for text analysis, and image sentiment and facial expression analysis for GIFs, with each frame individually analyzed. Our research shows that combining textual and picture data yields superior outcomes compared to models that depend only on either images or text. The output scores from our text, picture, and GIF modules will be aggregated to get the final sentiment score for the incoming tweets.