MLASK: Multimodal Summarization of Video-based News Articles
Mateusz Krubiński, Pavel Pecina · 2023
In recent years, the pattern of news consumption has been changing.The most popular multimedia news formats are now multimodal -the reader is often presented not only with a textual article but also with a short, vivid video.To draw the attention of the reader, such videobased articles are usually presented as a short textual summary paired with an image thumbnail.In this paper, we introduce MLASK 1 (MultimodaL Article Summarization Kit)a new dataset of video-based news articles paired with a textual summary and a cover picture, all obtained by automatically crawling several news websites.We demonstrate how the proposed dataset can be used to model the task of multimodal summarization by training a Transformer-based neural model.We also examine the effects of pre-training when the usage of generative pre-trained language models helps to improve the model performance, but (additional) pre-training on the simpler task of text summarization yields even better results.Our experiments suggest that the benefits of pre-training and using additional modalities in the input are not orthogonal.