Intent‐Based Versus GPT ‐Based Conversational Agents: Benefits and Challenges for Practicing and Assessing Oral Interaction
Veronika Timpe‐Laughlin, Rahul R. Divekar, Tetyana Sydorenko, Judit Dombi, Saerhim Oh · TESOL Quarterly · 2025
Abstract Interactional competence (IC) is crucial for L2 learners to communicate effectively across diverse contexts (Roever [2022], Teaching and testing second language pragmatics and interaction: A practical guide). However, providing opportunities to practice interactive oral skills is time and resource intensive, and assessing IC is challenging due to the need for diverse contexts and interlocutors to capture the dynamic nature of communication. Spoken dialogue systems (SDS) can act as interlocutors, enabling learners to demonstrate linguistic skills, but often result in rigid, transactional exchanges (Timpe‐Laughlin et al. [2024], Computer Assisted Language Learning , 37, 149–178). Large language models (LLMs), such as ChatGPT, which are purportedly capable of generating human‐like responses (Kostka & Toncelli [2023], TESL‐EJ , 27(3), 1–19), may offer greater flexibility, enabling interactive conversations that could effectively elicit phenomena of IC. In this study, we explored interactional phenomena in the oral output of 50 participants who engaged in the same role‐play task with an SDS and an LLM interlocutor, respectively. Participants' interactions were audio‐recorded and analyzed for interactional features, including openings and closings, repairs, small talk, and recipient design. Findings revealed that while both systems elicited IC phenomena, participants engaged in significantly more repair sequences and recipient design in SDS interactions, likely due to the system's more constrained processing capabilities. By contrast, the LLM played a more active role in carrying the conversation, making inferences about participant intent while leaving fewer opportunities for participants to show their IC. These differences highlight distinct interactional affordances of each system. We discuss the affordances and limitations of both systems for practicing and assessing oral IC skills, suggesting a hybrid approach to move ahead.