Unsupervised Emotion Matching for Image and Text Input
Inderpreet Singh Makkar, Shikha Chadha · 2024
As digital media continues to expand, accurately interpreting emotions from images and text poses an ongoing challenge. This paper introduces a novel unsupervised learning framework that synergistically combines convolutional neural networks and linguistic analysis to recognize emotions in multimedia without the need for labeled data. Our approach utilizes transfer learning and pre-trained emotion recognition models to create a robust emotion-matching algorithm. Considering the textual and visual inputs, the predicted emotion classes are compared pair-wise, and the results are analyzed qualitatively and quantitatively. The multimodal assessment in our study demonstrated a notable success rate of 84.25%, underscoring the robustness of our approach in handling and interpreting diverse data types. It has potential applications in social media, educational technology, and customer service, providing new avenues for understanding and interacting with digital content on an emotional level.