THE PROSPECTS OF USING TRANSFORMER-BASED MODELS IN NATURAL LANGUAGE PROCESSING TASKS
Andrei P. Titov, Nataliya V. Grishina, Dar’ya N. Titova · RSUH/RGGU Bulletin Series Information Science Information Security Mathematics · 2025
The article considers the architecture, operating principles and key features of T5 and BART, as well as their use in various NLP tasks. In recent years, models based on the T5 (Text-to-Text Transfer Transformer) and BART (Bidirectional and Auto-Regressive Transformers) transformers have evolved significantly in the field of natural language processing (NLP). Models are powerful tools for solving a wide range of tasks, including text generation, translation, summation, and classification. The T5 model is a universal one that converts all NLP tasks into the input text-output text format. That allows using the same architecture for different tasks, which greatly simplifies the process of learning and adapting the model. It is trained on a large dataset, which allows processing text data. The BART model combines elements of both auto – regressive and bidirectional models. It is trained on the task of text recovery. That allows it to effectively handle tasks related to text generation and conversion. BART demonstrates outstanding results in sampling and translating tasks due to its ability to take into account the context and structure of the text. The article goes into the advantages and disadvantages of both models and their impact on NLP development. It also considers prospects for further research. Their versatility and effectiveness make them important tools for researchers and practitioners working in that rapidly evolving field. The article studies the basic principles of operation of these models and their use in various tasks. Concrete examples demonstrate how T5 and BART overcome the limitations of previous approaches typical of traditional recurrent neural networks and other text processing methods. The study provides an in-depth analysis of the effectiveness and scalability of transformers, and also shows their contribution to the development of modern artificial intelligence systems. The authors emphasize that further improvements in the field of transformers can significantly expand the horizons of machine learning and natural language processing.