Trusted Multimodal Socio-Cyber Sentiment Analysis Based on Disentangled Hierarchical Representation Learning
Guoxia Xu, Lizhen Deng, Yansheng Li, Yantao Wei, Xiaokang Zhou, Hu Zhu · IEEE Transactions on Computational Social Systems · 2023
The rapid development of the digital age has led to a qualitative leap in social media. To meet the cognitive needs of users, social media platforms have been mining users’ private information and disseminating information through various means. However, these platforms lack effective management of information release and various forms of emotional expressions make public propaganda increasingly diverse and complex. Therefore, accurately identifying the relationships between multimodal data poses a challenge. An effective modal representation must consider both the consistency of multimodal data and the complementarity of single-modal data. However, existing methods focus on fusing different modal features into a unified feature representation, while neglecting to evaluate the reliability of prediction results. In this article, we disentangle the consistency and complementarity in the fused representation problem of multimodal data. We construct the modal private task (unique) by using the Dirichlet distribution and evidence theory to solve the uncertainty of each modal prediction. The model can output the uncertainty of prediction and learn complementary information through the fusion of decision layers. At the same time, we construct the modal common task using a low-rank tensor fusion model to learn consistent features. Finally, we compare the model with the current mainstream methods on three public datasets, and the experimental results show that the performance of our method reaches the level of current advanced algorithms.