A Performance Evaluation of Transformer Models and Recurrent Neural Networks Models in Efficient Text Classification Tasks
Babu Rao K, B Eswar Babu, Sathya Prakash Racharla, Pasupuleti Pavani, Yadaiah Balagoni, Mohan Ajmeera · 2025
This study provides a comparative evaluation of transformer models and Recurrent Neural Networks (RNNs) for text classification. As the complexity of the text data is growing and more precise classification mechanisms are required, both transformers and RNNs have received growing attention. Due to their attention mechanism, transformers are renowned for their capability of handling long-range dependencies and parallelizing the computations, thus they are especially beneficial for large-scale tasks. Conversely, RNNs are specifically built to handle sequential data and are renowned for their success in handling time-series data. They have difficulty with long-range dependencies because of vanishing or exploding gradients. This paper investigates the strengths and weaknesses of both paradigms through empirical experiments on standard datasets. Findings indicate that transformer models like BERT and GPT perform better than conventional RNN models in terms of accuracy and computational complexity, especially on longer text sequence datasets. The study also emphasizes the promise of hybrid methods and the way forward with domain-specific adaptation, such as domain-specific pre-training and knowledge distillation methods. The research indicates that though transformers are now superior in terms of performance, RNN-based models remain relevant to some text classification tasks, particularly in environments where resources are limited.