Revisiting non-English Text Simplification: A Unified Multilingual Benchmark
Michael Ryan, Tarek Naous, Wei Hong Xu · 2023
Recent advancements in high-quality, largescale English resources have pushed the frontier of English Automatic Text Simplification (ATS) research.However, less work has been done on multilingual text simplification due to the lack of a diverse evaluation benchmark that covers complex-simple sentence pairs in many languages.This paper introduces the MULTI-SIM benchmark, a collection of 27 resources in 12 distinct languages containing over 1.7 million complex-simple sentence pairs.This benchmark will encourage research in developing more effective multilingual text simplification models and evaluation metrics.Our experiments using MULTISIM with pre-trained multilingual language models reveal exciting performance improvements from multilingual training in non-English settings.We observe strong performance from Russian in zero-shot crosslingual transfer to low-resource languages.We further show that few-shot prompting with BLOOM-176b achieves comparable quality to reference simplifications outperforming finetuned models in most languages.We validate these findings through human evaluation.