Comparative Analysis of Transformer Models on WikiHow Dataset
Dev Jadeja, Avinash Khetri, Anubhav Mittal, Dinesh Kumar Vishwakarma · 2022 International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) · 2022
Because of the ever-increasing amount of information around us, it has become highly essential to consume it in the most efficient way as possible. Text Summarization is a way of generating a summary for a long piece of text while keeping the context and the meaning conveyed intact. Unlike Extractive Text Summarization, Abstractive Text Summarization is the process in which the summary is generated in a more humanlike way, without directly selecting a few sentences from the original piece of text itself. This research study has conducted a comparative analysis of T5 and PEGASUS, which are state-of-the-art abstractive text summarizers. The proposed study has used WikiHow Dataset for the comparison because its summaries are evenly extracted. The evaluation metrics used are BLEU and ROUGE. Unlike other researches doing comparisons, our novelty is in human evaluation for a subset of the summaries to see how reliable the evaluation metrics really are for abstractive summarization and in the use of the WikHow Dataset.