Vocabulary, corpus and language teaching. A machine-generated literature overview
Paweł Szudarski · ELT Journal · 2025
The launch of ChatGPT in November 2022 can be classified as a pivotal moment, one that compels us to pause and consider some fundamental questions related to our activities as educators and ELT professionals. What is learning and teaching all about? What is the role of technology in this process? How can technological tools enhance our daily work as language learners and teachers, materials writers, advisors, and policymakers? Such ponderings have been at the centre of the hype around ChatGPT, other chatbots, and artificial intelligence (AI) more broadly. Regardless of our primary focus in ELT, the technological revolution concerns all of us, and as such, it is understandable to see so much attention being paid to AI and its impact not only on academic disciplines but on other aspects of life as well. As regards education and ELT, it is enough to mention AI-focussed conferences (e.g., Global AI Forum 2024), special issues of journals (e.g., generative AI was the focus of special issues of ELT Journal, Studies in Higher Education, Language Learning & Technology, all published in 2024), and numerous informal conversations I am sure we all have had in the past two years, be it with colleagues (‘Is the essay that I am currently marking AI-generated?’) or friends (‘How can I use ChatGPT to make my emails in English sound better?’). I personally became aware of generative AI soon after the launch of ChatGPT. I would have heard about it through news reports and familiarised myself with some initial guidelines on AI published by my university. Out of curiosity, I also tested this tool myself, checking for instance its ability to write bespoke emails based on specific prompts. I must say I found it impressive how quickly ChatGPT produced new content, particularly when compared to the amount of time required to produce human (or ‘analogue’ for want of a better term) writing. However, in terms of published academic work, it was not until I came across Vocabulary, Corpus and Language Teaching, the machine-generated publication that is under review in this piece, that I realised the potential for disruption and the scope for misuse of AI during this new era of postplagiarism (Eaton 2023). As openly stated by Udaya and Reddy, their book was developed by means of an algorithm, seeking to ‘help young researchers to activate their knowledge in less time’ (p. 1). Specifically, using journals published by Springer Nature, each content chapter offers an auto-summary of selected papers addressing a given area of vocabulary studies. This decision is puzzling to say the least, because we know that generative AI tools enable the creation of new texts, images, or data, which then with some human curation and overseeing can potentially serve as a starting point for generating new knowledge. But if a book offers nothing but unchecked auto-summaries spewed out by a computer, I struggle to see how this makes an original contribution to knowledge and scholarship. As Yeo (2024) rightly observes, what is the point of publishing AI-generated summaries if readers themselves can produce them easily on their own with the help of ChatGPT or Co-Pilot? Thus, in what follows, I first discuss Vocabulary, Corpus and Language Teaching in terms of its structure and contents. Next, I use this description to offer a wider discussion of some of the ethical and pedagogical issues around the use of AI in academic writing, finishing with a call for a proactive response from all stakeholders in ELT, education and beyond. The book has six chapters, with Chapter 1 being an introduction and Chapter 6 serving as a very brief conclusion. Chapters 2-5 constitute the main body of the book, devoted, respectively, to Vocabulary and Acquisition (Chapter 2), Vocabulary Teaching and Learning Strategies (Chapter 3), Information and Communication Technology for Vocabulary Learning (Chapter 4), and Corpus-based Vocabulary (Chapter 5). Seeing this table of contents, I was quite looking forward to reading a systematised and up-to-date literature review of vocabulary studies, which as a field has grown significantly in the last 20-30 years. Unfortunately, and in many respects shockingly, this is not what I found. For one, the title of the publication is misleading. Reading ‘Vocabulary, Corpus and Language Learning’ on the cover, I thought the intention was to showcase empirical work at the interface of vocabulary studies and corpus linguistics, mirroring some existing publications with similar goals (Gardner 2013; Szudarski 2023). Disappointingly, that’s not the case here, as practically only Chapter 5 can be classified as relating to corpus-based lexical research. The other chapters, while addressing vocabulary topics, make no reference to corpora and their applications in lexical studies. In this sense, the title does not align with the declared aim of the publication, which the authors claim is ‘to help ESL/EFL researchers and teachers understand the domain of vocabulary instruction in the field of language education and learning’ (p. 1). Disconcertingly, this mismatch is not the main problem with this publication. What is far more concerning is what the individual chapters actually present. From the perspective of the reader, all the chapters provide is a set of seemingly random sentences touted as ‘the most important sentences’ in the original texts, if we follow the phrasing of the disclaimer at the beginning of each chapter. How this importance was determined is not explained. Further, as we learn from the introduction, some curation of these auto-generated summaries was apparently involved so that ‘they meet Springer Nature publication standards’ (p. 7), but the extent of this involvement seems minimal. Relatedly, despite the claim that the content was carefully edited, at the beginning of each chapter comes a caveat that ‘auto-selected sentences may not fully reflect the body of the work’, and as such, the editors ‘strongly advise that the original content is read and cited’ (p. 7). This begs at least two questions: about the quality of such machine-based summaries and about the purpose they are supposed to serve. Further, and more fundamentally perhaps, what does this publication say about academic integrity and the role of editors who participated in the curation of the book? These concerns become even more obvious if we consider that the publication is to be treated as an introduction to the field of vocabulary studies. If anything, oversimplified summaries produced by AI, when taken at face value, can easily misinform inexperienced or unfamiliar readers and lead them to assume that if this material was published, it must constitute good research. To illustrate these problems in more detail, I will refer to Chapter 3, ‘Vocabulary Teaching and Learning Strategies’ as an example. First, the chapter reports ten original studies related to learning strategies and vocabulary, but there is no one consistent way of discussing these studies. While headings such as ‘introduction’, ‘methodology’, or ‘results’ are included for each reviewed study, the amount of information presented under each of these headings differs considerably; for example, for some studies, all research questions are offered, for others only selected ones are provided or there is no information in this regard at all. Second, the amount of methodological detail varies significantly across the reviewed studies. By way of example, Section 3.1.6 reports a meta-analytic study into the effectiveness of L2 vocabulary instruction. Since meta-analysis is a systematic method that statistically combines the results of several studies and offers a general conclusion, this is a relevant topic that merits the attention of vocabulary scholars. However, the way this methodology is discussed in this particular case is misleading, presenting only selected details of the original design and therefore undermining the integrity of the original work. I can easily imagine someone reading this summary, particularly someone unfamiliar with this methodology, and assuming that conducting a meta-analysis is a fairly easy endeavour, where the selection of methodological criteria is optional or even random. Lastly, out of ten studies discussed in Chapter 3, only one explicitly addresses the practical implications of the reported research. There is virtually no attention being paid to its potential limitations, which I would see as a key feature of an effective literature review. Unfortunately, similar issues and reservations apply to the remaining chapters as well. Aside from the content, there are also problems with grammar, referencing conventions, and the overall coherence of the chapters. For instance, the following unfinished sentence opens one of the paragraphs in the Introduction (‘In Part II the contrast to “GroupLens” collaborative filtering algorithm’, p. 2). Further, no proper citations are included (e.g., ‘Ryan and Deci 36’, p. 3) and the text is not devoid of grammatical mistakes (‘the summaries in orient themselves’, p. 1 or ‘corpora have a direct implications for classroom instructions’, p 4). Clearly, all of this points to the lack of the human touch in this publication, one of the major risks often brought up in the characterisation of AI-supported work (Hadan et al. 2024). In short, Vocabulary, Corpus and Language Teaching provides 145 pages of computer-generated output, with very little (if any) evidence of human curation or critical appraisal during the process of writing. Such practice cannot be accepted, in my view, because it sets a dangerous precedent for the world of publishing. Further, such misuses of AI, and I dare say that we might see more examples of those in the wake of the AI revolution, also raise a number of important ethical and pedagogical questions for all of us to consider. And with technological development progressing so rapidly, it is perhaps unsurprising that the current state of knowledge poses more questions rather than provides answers, but this makes it even more pertinent to consider the pros and cons of AI (see Moorhouse 2024 for a useful overview in relation to ELT). In this light, I would like to end this review with a brief discussion of the impact of AI and its responsible use, encouraging more dialogue in the field and calling for action from all stakeholders involved in the processes of learning, teaching, assessment, and academic writing. Personally, I do not consider myself to be an expert and would rate my AI literacy as inadequate. But I do know what constitutes good academic writing, and what level of quality is expected from published work. It is certainly not what we find in Vocabulary, Corpus and Language Teaching. Yet, the problem is that this publication is already out in the world, signed off by a reputable publisher, and presented as a book to be consulted by individuals interested in vocabulary research. Thus, one of the first key points to consider is what is needed to prevent such fully machine-generated outputs from appearing as original academic outputs. With chatbots creating new content instantaneously and almost effortlessly, the temptation to produce ‘AI-enhanced’ work is only going to grow, particularly in the academic culture of publish-or-perish but in other contexts as well. Self-promotion and individual accomplishment might also take priority over the collective good of academia or society at large (Lewandowska-Tomaszczyk 2023). So how we can counteract such negative trends? Given the importance of writing in academic work, how will AI change the notions of authorship, creativity, originality, plagiarism, ownership? If there is evidence already that peer reviewers struggle to distinguish between human and AI-augmented writing (Hadan et al. 2024), how can we ensure academic integrity, transparency, and accountability? Where do we draw the line between ethical and unethical uses of AI? As Pack and Maloney (2024) argue in their fascinating discussion of AI applications in language education, answers to these questions can be relative depending on one’s perspective (e.g., students vs. teachers) and ‘it is unlikely that educators will all agree as to what constitutes ethical use’ (Pack and Maloney 2024:1013). If AI is here to stay, which it is, my view is that the question is not whether to allow its use (it is already happening), but how to use it, and how to use it responsibly and ethically, while maximising its potential. I would argue that given the enormity of this task, we all have an important part to play in our respective contexts and roles: as individual users of chatbots, teachers, supervisors, editors, reviewers, directors of assessment, and publishers. Humans are responsible for how AI tools are developed and used, and it is humans, not technology, that should account for ‘the accuracy, validity, reliability, and trustworthiness of scientific and scholarly outputs’ (Eaton 2023: 5). As we embrace this new AI-supported reality, the key step seems to be educating ourselves and raising awareness of the benefits and limitations of AI. As Pack and Maloney (2024) stress, it is essential that as language educators, we are proactive in developing our own digital competence and educate other teachers, students, and administrators about the potential (mis)use of AI. It is critical thinking and academic judgement that need to be at the forefront of this work, with the human touch seen as the critical element of AI literacy. As the reviewed book concerns corpus-based lexical research, a useful case in point is Curry, Baker, and Brookes (2024), a recent study into the applicability of ChatGPT for automated qualitative analysis of corpus-derived language data. They found that while ChatGPT dealt reasonably well with semantic-based categorisations of decontextualised words, it performed poorly with a more granular and context-dependent analysis of individual lines of text or function-to-form matches (e.g., failing to identify direct vs. indirect questions). Pointing to the issues of repeatability and replicability around such AI-based analysis, Curry, Baker, and Brookes 2024: 2) conclude that ‘ChatGPT is presently unable to meet the standards of the human analyst’, so close supervision and scrutiny of results are required for any AI-supported work. Such findings clearly underline the importance of developing AI competence, not only for research but also for teaching purposes. For instance, in the case of coursework and assessment, the presence of AI forces us as teachers to consider the diversification of tasks and assignments that we give to our students, with both creativity and transparency around the inclusion of AI as two conditions lying at the centre of AI-enhanced learning (for examples, see Nerantzi et al. 2023). With limited time available for professional development, it is also evident that teachers and language practitioners will need practical guidance and steer around effective use of AI, so it is encouraging to see resources such as Peachy and Crichton (2024), with a clear focus on classroom applications of AI. What is equally important is more research into the use of AI, its capabilities, sustainability, and long-term effects and risks. In relation to L2 learning and teaching specifically, a good illustration is Edmett et al.’s (2023) recent report written for the British Council, offering a global perspective with data collected from over 1,300 teachers in 118 countries. Such an ambitious global perspective is essential not only to inform the training and professional development of teachers that is needed, but also to prevent deepening divides between the digital haves and have-nots (the Global North vs. South). Further, while valuable as initial evidence, it is fair to say that most of the extant research has focused mainly on the perceptions of AI, rather than its effects on learning and measurable outcomes (Stockwell 2024). This is where research efforts need to be intensified. Much of the responsibility also falls on the shoulders of journal editors, reviewers, publishers, and professional organisations, with their clout and the gatekeeping functions they perform in the field as a whole. They should take the lead in producing policies and recommendations that will gradually filter down to individual writers, reviewers, consumers of research, and teachers (see Yeo 2024 for an effective and thought-provoking demonstration of the disruptive potential of generative AI from the perspective of a journal editor writing an editorial). The genie is out of the bottle. AI is here to stay and tools such as ChatGPT or Co-pilot already have a significant impact on what is happening in education and other aspects of life. My view is that as we face this new AI-supported reality, and more machine-generated or AI-augmented outputs such as Vocabulary, Corpus and Language Teaching are likely to appear, it is our collective responsibility as ELT professionals to consider what skills, knowledge, and policies are required to ensure research integrity and the maintenance of high standards in written and published material. I hope this review makes a contribution in this regard as well, highlighting some of the key issues around the use of AI, particularly in relation to research, scholarship, and academic writing. For a field where all these activities are highly relevant and likely to be affected by the presence of AI, it is our responsibility to encourage more dialogue and reflection around this quickly evolving situation. Paweł Szudarski is an Associate Professor of Applied Linguistics at the University of Nottingham, UK. He works in the areas of second language acquisition, corpus linguistics, and TESOL, focussing in particular on the acquisition of vocabulary and phraseology by second/foreign language learners. Paweł is also interested in replication research, digital pedagogies, and the teaching and learning of international students. Examples of his recent publications include Digital teaching for linguistics (Gregory et al. 2022) and Collocations, corpora and language learning (Szudarski 2023).