RIT Boston at SemEval-2022 Task 5: Multimedia Misogyny Detection By Using Coherent Visual and Language Features from CLIP Model and Data-centric AI Principle
Lei Chen, Hou Wei Chou · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · 2022
Detecting MEME images to be misogynous or not is an application useful on curbing online hateful information against women.In the SemEval-2022 Multimedia Automatic Misogyny Identification (MAMI) challenge, we designed a system using two simple but effective principles.First, we leverage on recently emerging Transformer models pretrained (mostly in a self-supervised learning way) on massive data sets to obtain very effective visual (V) and language (L) features.In particular, we used the CLIP (Radford et al., 2021) model provided by OpenAI to obtain coherent V and L features and then simply used a logistic regression model to make binary predictions.Second, we emphasized more on data rather than tweaking models by following the data-centric AI principle.These principles were proven to be useful and our final macro-F1 is 0.778 for the MAMI task A and ranked the third place among participant teams.