Citation Sentiment Classification using XLNet & Transformer Models on a Comprehensive Corpus
Mian Muhammad Danyal, Afsheen Khalid, Sarwar Shah Khan, Jamal Shah · Journal of Independent Studies and Research - Computing · 2025
The analysis of citation sentiment has emerged as an important research direction for understanding the polarity of scholarly communication beyond citation counts. Existing studies are often limited by class imbalance in available datasets and the lack of robust models that can capture the complex linguistic patterns of academic writing. To address these gaps, an enriched dataset of approximately 14,000 annotated citation texts was constructed by combining existing resources with additional manually curated data. Transformer-based architectures, though widely applied in other natural language processing tasks, have seen limited use in citation sentiment analysis. XLNet, a permutation-based transformer model, effectively captures bidirectional dependencies in scholarly texts, making it highly suitable for citation sentiment classification. Several baseline machine learning models, such as Linear Support Vector Ma- chines (LSVM) and Multinomial Na¨ıve Bayes (MNB), were also implemented for comparison. The experimental results demonstrate that transformer models significantly outperform traditional baselines, with the XLNet-based model achieving the best overall performance in terms of accuracy and F1-score. This research contributes by providing a stronger benchmark for citation sentiment analysis and enhancing the reliability of bibliometric indicators.