Multimodal Sentiment Analysis for Interviews and Proctoring
Namitha Rayasam, VV Sai Sridhar, NS Pushkar, Om Mathur, Anushka Ghei, Dhriti Rajesh Krishnan, K. Srinivas · 2024
This work presents a multimodal emotion analysis model that combines voice, facial, and textual analysis techniques for interviewing and proctoring use-cases. It leverages Convolutional Neural Networks (CNNs) for emotion detection and exam proctoring. The model combines CNN models with algorithms for voice stress analysis, facial emotion recognition, and finding speech patterns. This enables accurate classification of emotions such as sadness, happiness, surprise, anger, and others. In addition to enhancing online exam monitoring by detecting stress, anxiety, and deceptive behavior, the system utilizes Natural Language Processing (NLP) for text sentiment analysis and speech analysis algorithms to extract subtle emotional cues from spoken content. Experimental results show that our approach significantly improves emotion analysis models across various applications like healthcare, education, and human-computer interaction.