NOVEL TRANSFORMER-BASED APPROACH ENHANCED BY REINFORCEMENT LEARNING AND ATTENTION MECHANISMS
Mostafa Gamal, Mostafa Gamal, Hesham F. A. Hamed, Mustafa Abdul Salam, Sara Sweidan · Journal of Southwest Jiaotong University · 2023
In natural language processing (NLP), crafting succinct and precise text document summaries is a formidable challenge. Abstract text summarization (ATS) aims to extract the essence of the source text while preserving its main content and purpose. With the manual summarization of extensive text volumes posing a laborious task, the need for automated summarization techniques becomes increasingly evident. This research aims to pioneer an ATS model grounded in the transformative capabilities of the Transformer framework, enriched by a self-attention mechanism designed to tackle conference complexities, thus bolstering textual comprehension. As our journey reaches its zenith, we harness the empowering forces of reinforcement learning to elevate and refine the quality of our generated summaries. The resultant model exhibits remarkable advancements in text summarization performance. Trained and rigorously evaluated on a consolidated dataset comprising Inshorts News, DUC-2004, and CNN/Daily Mail shared tasks data, our model undergoes scrutiny using Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics, offering compelling evidence of its superiority over state-of-the-art baseline models. Impressively, our model attains formidable model accuracy, boasting a 49.10% F1-Score on both the Inshorts and CNN/Daily Mail datasets, along with a commendable 39.98% F1-Score on the DUC-2004 news dataset. Keywords: Natural Language Processing, Transformer Model, Self-Attention Mechanism, Abstract Text Summarization, Recall-Oriented Understudy for Gisting Evaluation Metrics DOI： https://doi.org/10.35741/issn.0258-2724.58.6.5