An Inquirer-Responder Architecture Using LLMs: Emulating Virtual Hearing Q&A in Education
Takafumi Nakanishi · 2023
This paper presents an Inquirer-Responder architecture consisting of multiple LLM instances and applies it to a method for automatically generating feedback comments on student reports by emulating virtual hearing Q&A. This method uses faculty education materials and student reports as inputs for multiple LLM instances. In education, quickly returning personalized feedback to each student is crucial but burdensome for faculty. The implementation of a system that automatically and promptly returns feedback on student submissions can improve student learning efficiency and reduce the faculty workload. Our proposed architecture consists of two LLM instances, referred to as inquirer and responder. The inquirer is an LLM instance that reads the material (e.g., lecture materials) provided by the faculty, poses questions, and evaluates the responses from the responder. The responder is an LLM instance that reads student submissions (e.g., reports) and answers questions posed by the inquirer based on the contents of those submissions. This architecture uses faculty materials and student submissions to emulate a virtual hearing, with evaluation based on the knowledge extracted from both the faculty materials and student submissions. This architecture can be applied to various systems that automatically and quickly check submissions and return feedback. In this study, we established a hypothetical use case for a university class and conducted experiments. As a result, there were some cases in which the evaluation was difficult owing to the performance of the current LLM. However, the proposed architecture provides a new perspective on the design of automatic feedback systems and is a step toward their improvement.