Noisy Text Data: foible of popular Transformer based NLP models
Kartikay Bagla, Shivam Gupta, Ankit Kumar, A. K. Gupta · 2023
In the past few years, researchers working in natural language processing have created a number of new models based on transformer architecture. These models have shown remarkable performance for various NLP tasks on benchmark datasets, often surpassing SOTA results. Buoyed with these results, industry practitioners are actively experimenting with fine-tuning these models to build NLP applications for industry use cases. However, for most datasets used by practitioners (to build these applications), it is hard to guarantee the complete absence of any noise in the data. While most transformer-based NLP models have performed exceedingly well in transferring the learnings from one dataset to another for various tasks, it remains unclear how these models perform when fine-tuned on noisy text. In this paper, we precisely do this.