Logically at the Constraint 2022: Multimodal role labelling
Ludovic Kun, Jayesh Bankoti, David Kiskovski · 2022
This paper describes our system for the Constraint 2022 challenge at ACL 2022, whose goal is to detect which entities are glorified, vilified or victimised, within a meme .The task should be done considering the perspective of the meme's author.In our work, the challenge is treated as a multi-class classification task.For a given pair of a meme and an entity, we need to classify whether the entity is being referenced as Hero, a Villain, a Victim or Other.Our solution combines (ensembling) different models based on Unimodal (Text only) model and Multimodal model (Text + Images).We conduct several experiments and benchmarks different competitive pre-trained transformers and vision models in this work.Our solution, based on an ensembling method, is ranked first on the leaderboard and obtains a macro F1-score of 0.58 on test set.The code for the experiments and results are available at here .