AStarTwice at SemEval-2021 Task 5: Toxic Span Detection Using RoBERTa-CRF, Domain Specific Pre-Training and Self-Training
Thakur Ashutosh Suman, Abhinav Jain · 2021
This paper describes our contribution to SemEval-2021 Task 5: Toxic Spans Detection.Our solution is built upon RoBERTa language model and Conditional Random Fields (CRF).We pre-trained RoBERTa on Civil Comments dataset, enabling it to create better contextual representation for this task.We also employed the semi-supervised learning technique of selftraining, which allowed us to extend our training dataset.In addition to these, we also identified some pre-processing steps that significantly improved our F1 score.Our proposed system achieved a rank of 41 with an F1 score of 66.16%.