Federated Meta-Learning for Emotion and Sentiment Aware Multi-modal Complaint Identification
Apoorva Singh, Siddarth Chandrasekar, Sriparna Saha, Tanmay Sen · 2023
Automatic detection of consumers' complaints about items or services they buy can be critical for organizations and online merchants.Previous studies on complaint identification are limited to text.Images along with the reviews can provide cues to identify complaints better, thus emphasizing the importance of incorporating multi-modal inputs into the process.Generally, the customer's emotional state significantly impacts the complaint expression; thus, the effect of emotion and sentiment on complaint identification must also be investigated.Furthermore, different organizations are usually not allowed to share their privacy-sensitive records due to data security and privacy concerns.Due to these issues, traditional models find it hard to understand and identify complaint patterns, particularly in the financial and healthcare sectors.In this work, we extend a benchmark multi-modal complaint dataset, a collection of reviews and images of the products posted on the e-commerce website Amazon.We propose a federated meta-learning-based multimodal multi-task framework for identifying complaints considering emotion recognition and sentiment analysis as two auxiliary tasks.Experimental results indicate that the proposed approach outperforms the baselines and the state-of-the-art approaches in centralized and federated meta-learning settings 1 .