Looking hard: Eye tracking for detecting grammaticality of automatically compressed sentences
Sigrid Klerke, Héctor Martínez Alonso, Anders Søgaard · DSpace repository (University of Tartu) · 2015
Natural language processing (NLP) tools are often developed with the intention of easing human processing, a goal which is hard to measure.Eye movements in reading are known to reflect aspects of the cognitive processing of text (Rayner et al., 2013).We explore how eye movements reflect aspects of reading that are of relevance to NLP system evaluation and development.This becomes increasingly relevant as eye tracking is becoming available in consumer products.In this paper we present an analysis of the differences between reading automatic sentence compressions and manually simplified newswire using eye-tracking experiments and readers' evaluations.We show that both manual simplification and automatic sentence compression provide texts that are easier to process than standard newswire, and that the main source of difficulty in processing machine-compressed text is ungrammaticality.Especially the proportion of regressions to previously read text is found to be sensitive to the differences in human-and computer-induced complexity.This finding is relevant for evaluation of automatic summarization, simplification and translation systems designed with the intention of facilitating human reading.