A Large Language Model-Based Autonomous Scoring Method for Subjective Question Answering under English Context
Ximeng Wen, Xinggan Fu · Journal of Circuits Systems and Computers · 2024
Scoring of subjective question answering costs much human labor in realistic applications. To deal with this issue, we propose to leverage the powerful semantic comprehension capability of the transformer-Bidirectional Encoder Representations from Transformers (BERT)-integrated model, and propose a large language model-based autonomous scoring method for subjective question-answering in the English context. First, a pretrained BERT structure is used to extract features from question sentences, obtaining vector representations of each word. Then, these vectors are input into the transformer encoder to obtain the semantic representation for the whole sentence. Next, the semantic content of subjective questions based on the context can be realized. Finally, we divide the student’s answers into different scoring intervals to obtain the final rating result. Besides, we compared the scoring results with other methods and used the Kaggle competition published dataset, NLPCC DBQA dataset, Text Retrieval Conference QA (TrecQA) dataset, and WikiQA dataset. The experimental results show that our proposal performs well in subjective English question scoring. Compared with traditional manual scoring, this method has achieved significant improvements in scoring efficiency and accuracy. In addition, we also conducted comparative experiments to demonstrate the effectiveness and applicability of the proposal. Through the powerful capabilities of the transformer-BERT-integrated model, the proposed model can accurately capture students’ writing ability and English proficiency.