Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns
Daniel Wiechmann, Yu Qiao, Elma Kerz, Justus Mattern · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2022
There is a growing interest in the combined use of NLP and machine learning methods to predict gaze patterns during naturalistic reading.While promising results have been obtained through the use of transformer-based language models, little work has been undertaken to relate the performance of such models to general text characteristics.In this paper we report on experiments with two eye-tracking corpora of naturalistic reading and two language models (BERT and GPT-2).In all experiments, we test effects of a broad spectrum of features for predicting human reading behavior that fall into five categories (syntactic complexity, lexical richness, register-based multiword combinations, readability and psycholinguistic word properties).Our experiments show that both the features included and the architecture of the transformer-based language models play a role in predicting multiple eye-tracking measures during naturalistic reading.We also report the results of experiments aimed at determining the relative importance of features from different groups using SP-LIME.