Influence of Language Proficiency on the Readability of Review Text and Transformer-based Models for Determining Language Proficiency
Salim Sazzed · Companion Proceedings of the Web Conference 2022 · 2022
In this study, we analyze the influence of the English language proficiency of non-native speakers on the readability of the text written by them. In addition, we present multiple approaches for automatically determining the language proficiency levels of non-native English speakers from the review data. To accomplish the above-mentioned tasks, we first introduce an annotated social media corpus of around 1000 reviews written by non-native English speakers of the following five English language proficiency (ELP) groups: very high proficiency (VHP), high proficiency (HP), moderate proficiency (MP), low proficiency (LP), and very low proficiency (VLP). We employ the Flesch Reading Ease (FLE) and Flesch-Kincaid Grade (FKG) tests to compute the readability scores of the reviews written by various ELP groups. We leverage both the classical machine learning (ML) classifiers and transformer-based language models for deciding the language proficiency groups of the reviewers. We observe that distinct ELP groups do not exhibit any noticeable differences in the mean FRE scores, although slight differences are observed in the FKG test. The results imply that the readability measures do not possess high discriminating capabilities to distinguish various ELP groups. In the language proficiency determination task, we notice fine-tuned transformer-based approaches yield slightly better efficacy than the traditional ML classifiers.