Contextualized Embedding based Approaches for Social Media-specific Sentiment Analysis
Harsh Sakhrani, Saloni Parekh, Pratik Ratadiya · 2021 International Conference on Data Mining Workshops (ICDMW) · 2021
Social media-specific Sentiment Analysis has a wide range of applications in various domains like Business Intelligence, Marketing, Politics and Psychology, to mention a few. Irony Detection and Emotion Recognition, two of Sentiment Analysis' significant pillars have become increasingly important as a result of the continued growth of social media. Previous approaches for the two tasks have yielded promising results, but have often relied on recurrence and pre-trained word- embedding ensembles. In this paper, we propose two novel contextual embedding-based approaches for Irony Detection and Emotion Recognition. We leverage social media-specific pre- training in the form of BERTweet - A language model pre-trained on English Tweets, along with either a Convolutional Neural Network or a Transformer Encoder. We empirically show that the addition of Convolutional Neural Networks or a Transformer Encoder results in improved performance when compared to a vanilla BERTweet model. Furthermore, we also compare CNNs and the Transformer Encoder as feature extractors, assessing the trade-off between the number of learnable parameters and performance. Finally, we also investigate the impact of partial and complete fine-tuning and analyze the trade-off between computational power and accuracy in the process. Experimental results demonstrate that our proposed methods achieve state-of- the-art performance on two standard benchmark datasets.