Enhancing Sentiment Analysis with Fine-Tuned DeBERTa in a Cross-Domain Study on IMDb and Amazon Polarity Datasets
Zijian Cao · Applied and Computational Engineering · 2025
Sentiment analysis has become crucial for understanding consumer behavior in e-commerce and social media. While models based on transformers like Bidirectional Encoder Representations from Transformers (BERT) and A Robustly Optimized BERT Pretraining Approach (RoBERTa) have great performance in Natural Language Processing (NLP) tasks, challenges persist in domain-specific language nuances and context sensitivity. This study addresses these gaps by utilizing DeBERTa, an advanced model with disentangled attention mechanisms, to improve sentiment classification. The methodology involves fine-tuning the DeBERTa-base on the IMDb dataset and evaluating its generalization on the Amazon Polarity dataset. The results achieved 93.21% accuracy, 91.28% precision, 94.98% recall, and 93.32% F1 score, achieving a good balance between precision and recall while maintaining high accuracy. The result of this study demonstrates that Decoding-enhanced BERT with Disentangled Attention (DeBERTa) has strong performance in handling long, structured text and short, diverse comments. This illustrates DeBERTa's superiority in capturing complex semantic relationships and context-dependent sentiments.