Toward Multimodal Complaint Severity Detection From Social Media
Apoorva Singh, Prince Jha, Souryadip Das, Raghav Jain, Sriparna Saha · IEEE Transactions on Computational Social Systems · 2024
The prevalence of complaints submitted online and the sheer volume of information made available by social media platforms highlight the need for automated complaint analysis tools. In linguistic studies, complaints have been classified according to how much of personal risk the complainant is willing to take. This is crucial information for understanding the motivations of complainants and how people come up with reasonable means of reparation. Few attempts have been made to use the existing multimodal complaint model, which focuses on improving the textual mode with the help of images, to find specific visual features that help identify complaints. Our aim is to find a solution to this problem. In order to detect complaints and the severity level associated with them in a multitask setting, we propose Multimodal framEwork for Complaint and Severity-level detectIon (MECSI), a novel multimodal framework that uses local and global attributes (in both modalities) and relates them to the textual context. To do this, we add severity-level annotation to the newly released CESAMARD dataset, which is a compilation of reviews and images of products listed on the Amazon website. The experimental findings confirmed the superiority of our proposed model over the state-of-the-art model and other strong rival baselines, proving the efficacy of our proposed framework.