Quality estimation for text simplification
Sanja Štajner, Maja Popović, Hanna Béchara · 2016
The quality of the output generated by automatic Text Simplification (TS) systems is traditionally assessed by human annotators. In spite of the fact that the automatisation of that process would enable faster and more consistent evaluation, there have been almost no studies addressing this problem. We propose several decision-making procedures for automatic classification of the simplified sentences into three classes (bad, OK, good) depending on their grammaticality, meaning preservation, and simplicity. We experiment with ten different classification algorithms and 12 different feature sets on three TS datasets obtained using different text simplification strategies, achieving the results significantly above the state of the art. Additionally, we propose to use an unique measure (Total2 or Total3) for classifying the quality of the automatically simplified sentences into two (discard or keep) or three (bad, OK, good) classes.