StanceCrafters at StanceEval2024: Multi-task Stance Detection using BERT Ensemble with Attention Based Aggregation
Ahmed Abul Hasanaath, Aisha Alansari · 2024
Stance detection is a key NLP problem that classifies a writer's viewpoint on a topic based on their writing.This paper outlines our approach for Stance Detection in Arabic Language Shared Task (StanceEval2024), focusing on attitudes towards the COVID-19 vaccine, digital transformation, and women's empowerment.The proposed model uses parallel multi-task learning with two fine-tuned BERT-based models combined via an attention module.Results indicate this ensemble outperforms a single BERT model, demonstrating the benefits of using BERT architectures trained on diverse datasets.Specifically, Arabert-Twitterv2, trained on tweets, and Camel-Lab, trained on Modern Standard Arabic (MSA), Dialectal Arabic (DA), and Classical Arabic (CA), allowed us to leverage diverse Arabic dialects and styles.The code has been made open-source on the link https://github.com/gufranSabri/StanceDetection- MultiTaskLearning.