A Generative Approach to Titling and Clustering Wikipedia Sections
Anjalie Field, Sascha Rothe, Simon Baumgartner, Cong Yu, Abe Ittycheriah · 2020
We evaluate the performance of transformer encoders with various decoders for information organization through a new task: generation of section headings for Wikipedia articles.Our analysis shows that decoders containing attention mechanisms over the encoder output achieve high-scoring results by generating extractive text.In contrast, a decoder without attention better facilitates semantic encoding and can be used to generate section embeddings.We additionally introduce a new loss function, which further encourages the decoder to generate high-quality embeddings.