Large Scale Multi-Lingual Multi-Modal Summarization Dataset
Yash Verma, Anubhav Jangra, Raghvendra Verma, Sriparna Saha · 2023
Significant developments in techniques such as encoder-decoder models have enabled us to represent information comprising multiple modalities.This information can further enhance many downstream tasks in the field of information retrieval and natural language processing; however, improvements in multi-modal techniques and their performance evaluation require large-scale multi-modal data which offers sufficient diversity.Multi-lingual modeling for a variety of tasks like multi-modal summarization, text generation, and translation leverages information derived from highquality multi-lingual annotated data.In this work, we present the current largest multilingual multi-modal summarization dataset (M3LS), and it consists of over a million instances of document-image pairs along with a professionally annotated multi-modal summary for each pair.It is derived from news articles published by British Broadcasting Corporation(BBC) over a decade and spans 20 languages, targeting diversity across five language roots, it is also the largest summarization dataset for 13 languages and consists of cross-lingual summarization data for 2 languages.We formally define the multi-lingual multi-modal summarization task utilizing our dataset and report baseline scores from various state-of-the-art summarization techniques in a multi-lingual setting.We also compare it with many similar datasets to analyze the uniqueness and difficulty of M3LS.