Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue
Alexios Gidiotis, Grigorios Tsoumakas · Findings of the Association for Computational Linguistics: ACL 2022 · 2022
We explore the notion of uncertainty in the context of modern abstractive summarization models, using the tools of Bayesian Deep Learning.Our approach approximates Bayesian inference by first extending stateof-the-art summarization models with Monte Carlo dropout and then using them to perform multiple stochastic forward passes.Based on Bayesian inference we are able to effectively quantify uncertainty at prediction time.Having a reliable uncertainty measure, we can improve the experience of the end user by filtering out generated summaries of high uncertainty.Furthermore, uncertainty estimation could be used as a criterion for selecting samples for annotation, and can be paired nicely with active learning and human-in-the-loop approaches.Finally, Bayesian inference enables us to find a Bayesian summary which performs better than a deterministic one and is more robust to uncertainty.In practice, we show that our Variational Bayesian equivalents of BART and PEGASUS can outperform their deterministic counterparts on multiple benchmark datasets.