Crowdsourced Corpus of Sentence Simplification with Core Vocabulary
Akihiro Katsuta, Kazuhide Yamamoto · 2018
We present a new Japanese crowdsourced data set of simplified sentences created from more complex ones.Our simplicity standard involves all rewritable words in the simplified sentences being drawn from a core vocabulary of 2,000 words.Our simplified corpus is a collection of complex sentences from Japanese textbooks and reference books together with simplified sentences generated by humans, paired with data on how the complex sentences were paraphrased.The corpus contains a total of 15,000 sentences, in both complex and simple versions.In addition, we investigate the differences in the simplification operations used by each annotator.The aim is to understand whether a crowdsourced complex-simple parallel corpus is an appropriate data source for automated simplification by machine learning.The results, that there was a high level of agreement between the annotators building the data set.So, we believe that this corpus is a good quality data set for machine learning for simplification.We therefore plan to expand the scale of the simplified corpus in the future.