UniBO at SemEval-2022 Task 5: A Multimodal bi-Transformer Approach to the Binary and Fine-grained Identification of Misogyny in Memes
Arianna Muti, Katerina Korre, Alberto Barrón‐Cedeño · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · 2022
We present our submission to SemEval 2022 Task 5 on Multimedia Automatic Misogyny Identification.We address the two tasks: Task A consists of identifying whether a meme is misogynous.If so, Task B attempts to identify its kind among shaming, stereotyping, objectification, and violence.Our approach combines a BERT Transformer with CLIP for the textual and visual representations.Both textual and visual encoders are fused in an early-fusion fashion through a Multimodal Bidirectional Transformer with unimodally pretrained components.Our official submissions obtain macroaveraged F 1 =0.727 in Task A (4th position out of 69 participants) and weighted F 1 =0.710 in Task B (4th position out of 42 participants).