Semantics Squad at BLP-2023 Task 1: Violence Inciting Bangla Text Detection with Fine-Tuned Transformer-Based Models
Krishno Dey, Prerona Tarannum, Md. Arid Hasan, Francis Palma · 2023
This study investigates the application of Transformer-based models for violence threat identification.We participated in the BLP-2023 Shared Task 1 and in our initial submission, BanglaBERT large achieved 5 th position on the leader-board with a macro F1 score of 0.7441, approaching the highest baseline of 0.7879 established for this task.In contrast, the top-performing system on the leaderboard achieved an F1 score of 0.7604.Subsequent experiments involving m-BERT, XLM-RoBERTa base, XLM-RoBERTa large, BanglishBERT, BanglaBERT, and BanglaBERT large models revealed that BanglaBERT achieved an F1 score of 0.7441, which closely approximated the baseline.Remarkably, m-BERT and XLM-RoBERTa base also approximated the baseline with macro F1 scores of 0.6584 and 0.6968, respectively.A notable finding from our study is the under-performance by larger models for the shared task dataset, which requires further investigation.Our findings underscore the potential of transformer-based models in identifying violence threats, offering valuable insights to enhance safety measures on online platforms.