StyleBART: Decorate Pretrained Model with Style Adapters for Unsupervised Stylistic Headline Generation
Hanqing Wang, Yajing Luo, Boya Xiong, Guanhua Chen, Yun Chen · 2023
Stylistic headline generation is the task of generate a headline that not only summarizes the content of a news, but also reflects a desired style that attracts users.As style-specific newsheadline pairs are scarce, previous research has focused on unsupervised approaches using a standard headline generation dataset and monostyle corpora.In this work, we follow this line and propose StyleBART, an unsupervised approach for stylistic headline generation.Our method decorates the pretrained BART model with adapters that are responsible for different styles and allows the generation of headlines with diverse styles by simply switching the adapters.Different from previous works, StyleBART separates the task of style learning and headline generation, making it possible to freely combine the base model and the style adapters during inference.We further propose an inverse paraphrasing task to enhance the style adapters.Extensive automatic and human evaluations show that Style-BART achieves new state-of-the-art performance in the unsupervised stylistic headline generation task, producing high-quality headlines with the desired style.Code is available at https://github.com/sufenlp/StyleBART.