Investigating the Use of Intelligent Tutors Based on Large Language Models: Automated generation of Business Process Management questions using the Revised Bloom's Taxonomy
Guilherme Rego Rockembach, Lucinéia Heloisa Thom · 2024
The construction of assessment artifacts is a complex task, since generating appropriate assessments manually requires in-depth knowledge of both the area to be assessed and the cognitive processes involved in learning. The use of Large Language Models (LLMs) as the basis for the operation of Intelligent Tutoring Systems can assist in this task. This work experiments with the GPT-3.5-Turbo and LLama-2 LLMs as a source of automatic generation of assessment questions. The experiment was carried out using Prompt Engineering techniques to generate questions for the Business Process Management (BPM) discipline. From the experiment, it was possible to observe that both models are capable of generating questions appropriate to the BPM context. It was also identified that, when it received the context and the model of the question to be generated, the LLama-2 model produced questions more appropriate to the desired cognitive level, while the GPT-3.5-Turbo model received only the context and produced a similar response.