Applying Transfer Learning to Sentiment Analysis in Imbalanced Citation Sentiment Analysis 11 Dataset Using BERT
Jename Tadlip, Jolina Monisit, Katherine Dequito, Takeyasu V. Nakazato, Regien B. Nakazato · 2024
A Deep Learning model based on Bidirectional Encoder Representation from Transformers (BERT) was used to perform handling imbalanced datasets in this study. Imbalanced datasets, where one class has significantly more data than the other, pose difficulties for accurate classification. The experiments involve training BERT on the balanced data and evaluating its performance, followed by fine-tuning on the imbalanced data to assess its ability to handle such datasets. Assessment criteria like accuracy, precision, recall, and F1 score are employed for gauging its effectiveness. The experimental results demonstrate that BERT achieves high accuracy when trained on imbalanced data, outperforming its performance on balanced data. The accuracy obtained during the fine-tuning process with the imbalance dataset achieves 98% while the accuracy rate for the pre-training process achieves 88%. This manuscript contributes to the understanding of BERT's effectiveness in sentiment analysis and highlights the need to address data imbalance for accurate classification. This emphasizes the importance of implementing data balancing techniques to enhance the effectiveness of models used for analyzing sentiment.