Inverse Reinforcement Learning for Text Summarization
Yu Ping Fu, Deyi Xiong, Yue Dong · 2023
We introduce inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models, imitating human summarization behaviors.Our IRL model estimates the reward function using a suite of important sub-rewards for summarization and concurrently optimizes the policy network.Experimental results across datasets in different domains (CNN/DailyMail and WikiHow) and various model sizes (BART-base and BARTlarge) demonstrate the superiority of our proposed IRL model for summarization over MLE and RL baselines.The resulting summaries exhibit greater similarity to human-crafted gold references, outperforming MLE and RL baselines on metrics such as ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations.