YNU-HPCC at SemEval-2022 Task 5: Multi-Modal and Multi-label Emotion Classification Based on LXMERT
Chao Han, Jin Wang, Xuejie Zhang · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · 2022
This paper describes our system used in the SemEval-2022 Task5 Multimedia Automatic Misogyny Identification (MAMI).This task is to use the provided text-image pairs to classify emotions.In this paper, We propose a multi-label emotion classification model based on pre-trained LXMERT.We use Faster-RCNN to extract visual representation and utilize LXMERT's cross-attention for multimodal alignment.Then we use the Bilinearinteraction layer to fuse these features.Our experimental results surpass the F 1 score of baseline.For Sub-task A, our F 1 score is 0.662 and Sub-task B's F 1 score is 0.633.The code of this study is available on GitHub 1 .