Contextual Inter-modal Attention for Multi-modal Sentiment Analysis
Deepanway Ghosal, Md Shad Akhtar, Dushyant Singh Chauhan, Soujanya Poria, Asif Ekbal, Pushpak Bhattacharyya · 2018
Multi-modal sentiment analysis offers various challenges, one being the effective combination of different input modalities, namely text, visual and acoustic.In this paper, we propose a recurrent neural network based multi-modal attention framework that leverages the contextual information for utterance-level sentiment prediction.The proposed approach applies attention on multi-modal multi-utterance representations and tries to learn the contributing features amongst them.We evaluate our proposed approach on two multi-modal sentiment analysis benchmark datasets, viz.CMU Multi-modal Opinion-level Sentiment Intensity (CMU-MOSI) corpus and the recently released CMU Multi-modal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus.Evaluation results show the effectiveness of our proposed approach with the accuracies of 82.31% and 79.80% for the MOSI and MO-SEI datasets, respectively.These are approximately 2 and 1 points performance improvement over the state-of-the-art models for the datasets.