Eye tracking as a measure of cognitive effort for post-editing of machine translation
Joss Moorkens · Benjamins translation library · 2018
Abstract The three measurements for post-editing effort as proposed by Krings (2001) have been adopted by many researchers in subsequent studies and publications. These measurements comprise temporal effort (the speed or productivity rate of post-editing, often measured in words per second at the segment level), technical effort (the number of actual edits performed by the post-editor, sometimes approximated using the Translation Edit Rate metric ( Snover et al. 2006 ), again usually at the segment level), and cognitive effort. Cognitive effort has been measured using think-aloud protocols, pause measurement, and, increasingly, eye-tracking. This chapter provides a review of studies of post-editing effort using eye-tracking, noting the influence of publications by Danks et al. (1997) , and O’Brien ( 2006 , 2008 ), before describing a single study in detail. The detailed study examines whether predicted affort indicators affect post-editing effort and results were previously published as Moorkens et al. (2015) . This chapter focuses instead on methodology and the logistics of running an eye-tracking study recording over 70 sessions. Most of the eye-tracking data analysed were unused in the previous publication, and the small amount presented was not described in detail due to space constraints. In this study average fixation count per segment correlates very strongly with temporal effort, and average fixation duration correlates strongly with technical effort, a result that we compare with other studies of post-editing effort.