Factual Error Correction for Abstractive Summarization Models
Meng Cao, Yue Dong, Jiapeng Wu, Jackie Chi Kit Cheung · 2020
Neural abstractive summarization systems have achieved promising progress, thanks to the availability of large-scale datasets and models pre-trained with self-supervised methods.However, ensuring the factual consistency of the generated summaries for abstractive summarization systems is a challenge.We propose a post-editing corrector module to address this issue by identifying and correcting factual errors in generated summaries.The neural corrector model is pre-trained on artificial examples that are created by applying a series of heuristic transformations on reference summaries.These transformations are inspired by an error analysis of state-of-the-art summarization model outputs.Experimental results show that our model is able to correct factual errors in summaries generated by other neural summarization models and outperforms previous models on factual consistency evaluation on the CNN/DailyMail dataset.We also find that transferring from artificial error correction to downstream settings is still very challenging 1 .Article: Jerusalem (CNN)The flame of remembrance burns in Jerusalem, and a song of memory haunts Valerie Braham as it never has before.(...) "Now I truly understand everyone who has lost a loved one," Braham said.Her husband, Philippe Braham, was one of 17 people killed in January's terror attacks in Paris.He was in a kosher supermarket when a gunman stormed in, killing four people, all of them Jewish.(...) Original: Valerie braham was one of 17 people killed in january's terror attacks in paris.(inconsistent) Corrected: Philippe braham was one of 17 people killed in january's terror attacks in paris.(consistent) Article: (...) Thursday's attack by al-Shabaab militants killed 147 people, including 142 students, three security officers and two university security personnel.The attack left 104 people injured, including 19 who are in critical condition, Nkaissery said.(...) Original: 147 people, including 142 students, are in critical condition.(inconsistent) Corrected: 19 people, including 142 students, are in critical condition.(inconsistent) Article: (CNN) Officer Michael Slager's five-year career with the North Charleston Police Department in South Carolina ended after he resorted to deadly force following a routine traffic stop.(...) His back is to Slager, who, from a few yards away, raises his gun and fires.Slager is now charged with murder.The FBI is involved in the investigation of the slaying of the father of four.(...) Original: Slager is now charged with murder.(consistent) Corrected: Michael Slager is now charged with murder.(consistent) Article: (CNN)The announcement this year of a new, original Dr. Seuss book sent a wave of nostalgic giddiness across Twitter, and months before publication, the number of pre-orders for "What Pet Should I Get?" continues to climb.(...) It features the spirited siblings from the beloved classic "One Fish Two Fish Red Fish Blue Fish" and is believed to have been written between 1958 and 1962.(...) Original: Seuss book sent a wave of nostalgic giddiness across twitter.(consistent) Corrected: "One Fish Two Fish Red Fish Blue Fish" book sent a wave of nostalgic giddiness across twitter.(inconsistent)