BERTweet: A pre-trained language model for English Tweets
Dat Quoc Nguyen, Thanh Vu, Anh Tuan Nguyen · 2020
We present BERTweet, the first public largescale pre-trained language model for English Tweets.Our BERTweet, having the same architecture as BERT base (Devlin et al., 2019), is trained using the RoBERTa pre-training procedure (Liu et al., 2019).Experiments show that BERTweet outperforms strong baselines RoBERTa base and XLM-R base (Conneau et al., 2020), producing better performance results than the previous state-of-the-art models on three Tweet NLP tasks: Part-of-speech tagging, Named-entity recognition and text classification.We release BERTweet under the MIT License to facilitate future research and applications on Tweet data.Our BERTweet is