Transformer Models for Drug Adverse Effects Detection from Tweets
Pavel Dmitrievich Blinov, Manvel Avetisian · 2020
In this paper we present the drug adverse effects detection system developed during our participation in the Social Media Mining for Health Applications Shared Task 2020. We experimented with transfer learning approach for English and Russian, BERT and RoBERTa architectures and several strategies for regression head composition. Our final submissions in both languages overcome average F1 by several percents margin.