RuSimScore: unsupervised scoring function for Russian sentence simplification quality
Saint Petersburg, Russia, Mikhail Orzhenovskii · Computational Linguistics and Intellectual Technologies · 2021
We propose an unsupervised complex scoring function (RuSimScore) to measure simplification quality of Russian sentences, and a model for text simplification based on this function.The function allows to score simplicity and original meaning preservation.First, filtered a noisy parallel corpus (machine translated WikiLarge) and extracted good simplification examples.After that, a pretrained language model was fine-tuned on these examples.We generate multiple outputs from the language model and select the best one according to the scoring function.The weights in the scoring function can be adjusted to balance between better content preservation and getting simpler sentences (controllable simplification).