UPB at SemEval-2021 Task 7: Adversarial Multi-Task Learning for Detecting and Rating Humor and Offense
Răzvan-Alexandru Smădu, Dumitru-Clementin Cercel, Mihai Dascălu · 2021
Detecting humor is a challenging task since words might share multiple valences and, depending on the context, the same words can be even used in offensive expressions.Neural network architectures based on Transformer obtain state-of-the-art results on several Natural Language Processing tasks, especially text classification.Adversarial learning, combined with other techniques such as multi-task learning, aids neural models learn the intrinsic properties of data.In this work, we describe our adversarial multi-task network, AMTL-Humor, used to detect and rate humor and offensive texts from Task 7 at SemEval-2021.Each branch from the model is focused on solving a related task, and consists of a BiL-STM layer followed by Capsule layers, on top of BERTweet used for generating contextualized embeddings.Our best model consists of an ensemble of all tested configurations, and achieves a 95.66% F1-score and 94.70% accuracy for Task 1a, while obtaining RMSE scores of 0.6200 and 0.5318 for Tasks 1b and 2, respectively.