RuSimpleSentEval-2021 Shared Task: Evaluating Sentence Simplification for Russian
Andrey Sakhovskiy, Alexandra Izhevskaya, Alena Pestova, Elena Viktorovna Tutubalina, Valentin Andreevich Malykh, Ivan Smurov, ABBYY, Moscow, Russia, Ekaterina Artemova · Computational Linguistics and Intellectual Technologies · 2021
This report presents the results from the RuSimpleSentEval Shared Task conducted as a part of the Dialogue 2021 evaluation campaign.For the RSSE Shared Task, devoted to sentence simplification in Russian, a new middlescale dataset is created from scratch.It enumerates more than 3000 sentences sampled from popular Wikipedia pages.Each sentence is aligned with 2.2 simplified modifications, on average.The Shared Task implies sequenceto-sequence approaches: given an input complex sentence, a system should provide with its simplified version.A popular sentence simplification measure, SARI, is used to evaluate the system's performance.Fourteen teams participated in the Shared Task, submitting almost 350 runs involving different sentence simplification strategies.The Shared Task was conducted in two phases, with the public test phase allowing an unlimited number of submissions and the brief private test phase accepting one submission only.The post-evaluation phase remains open even after the end of private testing.The RSSE Shared Task has achieved its objective by providing a common ground for evaluating state-of-the-art models.We hope that the research community will benefit from the presented evaluation campaign.https://github.com/dialogue-evaluation/RuSimpleSentEval/.