Exploring Transformers for Scalable Summarization of Long Documents
Mohammed Wasid, Mohd. Aquib Ansari, Aditya Pandey, Vanshagra Rai · 2025
The matter of document summarization pertains to the sub-field of Natural Language Processing or NLP, and it is instrumental in numerous industries and sectors. It can be used in many applications like journalism, research, legal documentation, etc. to quickly understand large documents without losing any relevant details. Advances in technology have so far brought transformer models to enrich the capabilities of long document summarization. Transformers are capable of capturing better dependencies from sequential information than other recurrent learning units like RNN and LSTM. Therefore, this paper is highly recommended for reviewing the architectures, approaches, strengths and weaknesses of top existing transformer models since it surveys the main existing models on this topic. A comparison of several transformer models, methodologies, and performance metrics is presented in tabular form in the review, which could help scientists and engineers engaged in this line of research.