Grading Programming Assignments by Summarization
Dong Dong, Yue Liang · 2024
Grading programming assignments manually is a big burden for instructors who teach programming languages for university students due to complexity and subjectivity. The black test approach adopted by online judge systems can only outputs either an answer is correct or incorrect. This study proposes a Large Language Model (LLM) approach to automatically grade answers from students for programming assignments. A LLM mode formed by coder-decoder architecture is utilized to generate summarization from source code, then the summarization is compared to the textual assignment description by semantic similarity. Finally, the output is converted to five-score rating. CodeBERT and a Transformer model serve as coder and decoder respectively. The semantic similarity is computed by MiniLM-L6. The validation test shows that the accuracy of the suggested approach reaches 0.92.