An Improved Model of Multi-attention LSTM for Multimodal Sentiment Analysis
Anna Wang, Baoshan Sun, Rize Jin · 2019
Multimodal sentiment analysis is the task of detecting emotions in videos using multimodal information such as text, visual and audio. One difficulty that is often faced is the complexity associated with different modes in the fusion of feature layers. In this paper, we present a novel feature-level fusion method for analyzing emotions called Multi-attention LSTM (MALM). The proposed approach uses LSTM to capture context information of contexts in the same video. At the same time, we use the attention mechanism before and after multimodal information fusion in order to focus attention on relatively important sequences and modalities. We evaluate our proposed approach on two multi-modal sentiment analysis benchmark datasets and compare to various proposed approaches on the same datasets. Evaluation results show approximately 5-10% performance improvement over the state-of-the-art models for the benchmark datasets.