AN EXPLORATORY ASSESSMENT OF THE USABILITY AND POTENTIAL OF GENERATIVE PRETRAINED TRANSFORMERS (GPTS) AS FEEDBACK ASSISTANTS FOR LONG-FORMAT ACADEMIC WRITING TASKS
Jeroen Lievens · INTED proceedings · 2024
This paper addresses the challenge faced by higher education practitioners in delivering precise and meaningful feedback on long-format written assignments, given the constraints of time and large student cohorts that are common in higher education. Taking its cue from recent advancements in artificial intelligence (AI), particularly GPT models like the large language model (LLM) ChatGPT4, this study investigates the potential of GPTs for providing in-depth feedback, including feedback relating to higher-order concerns such as methodological soundness, critical thinking, logic and originality. The methodology initially proposed involved the development of a dedicated ChatGPT4 agent, using OpenAI’s GPT Builder, for automated feedback purposes. However, due to access limitations on ChatGPT4, the study shifted, of necessity, to a literature review focusing on the feedback capabilities of LLMs. The investigation is structured around the SWOT analysis framework, exploring the strengths, weaknesses, opportunities, and threats associated with implementing LLMs for feedback purposes. Results from the literature review indicate several strengths of ChatGPT-based feedback such as its comparability to human feedback, readability, and potential to identify major issues. However, weaknesses are observed in the model's domain-specific feedback generation, consistency, and potential biases. Opportunities lie in future improvements and in collaborative human-AI approaches. Threats, finally, include overreliance on AI, issues of equity, data privacy concerns, biases, and the potential erosion of the human-learner relationship.