Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt
Xiaocui Yang, Feng Shi, Daling Wang, Qi Sun, Wenfang Wu, Yifei Zhang, Pengfei Hong, Soujanya Poria · 2023
We have witnessed the rapid proliferation of multimodal data on numerous social media platforms.Conventional studies typically require massive labeled data to train models for Multimodal Aspect-Based Sentiment Analysis (MABSA).However, collecting and annotating fine-grained multimodal data for MABSA is tough.To alleviate the above issue, we perform three MABSA-related tasks with quite a small number of labeled multimodal samples.We first build diverse and comprehensive multimodal few-shot datasets according to the data distribution.To capture the specific prompt for each aspect term in a few-shot scenario, we propose a novel Generative Multimodal Prompt (GMP) 1 model for MABSA, which includes the Multimodal Encoder module and the N-Stream Decoders module.We further introduce a subtask to predict the number of aspect terms in each instance to construct the multimodal prompt.Extensive experiments on two datasets demonstrate that our approach outperforms strong baselines on two MABSA-related tasks in the few-shot setting.