LLM-based Individual Contribution Summarization in Software Projects
Fabio de Miranda, Rafael Corsi Ferrão, Diego Pavan Soler, Marcelo Augusto Vieira Graglia · 2024
This work in progress is about preliminary results in using a Large Language Model (LLM) to summarize individual student contributions in open-ended software projects. Projects for industry clients are good real-world learning opportunities. Though, if the scope is open and defined based on external clients' needs, each group's project will look unique, what makes a challenge for grading and regular feedback. Distributed code version control systems such as Git and resources such as Git classroom help, but it is still burdensome to have professors and TAs looking at the repositories with a frequency that enables useful, timely feedback for the students. We prototyped a method of summarizing each student's contributions to a project's Git repository using an LLM, indicating how to preprocess and break down repository data in order to get better responses from the system. Each student's contributions were extracted using Pydriller. This technique was tested during a 3-week full-time software development sprint in a class of 28 students. Preliminary results indicate a general agreement of students and faculty with the synthesized summaries and an increase in students' awareness of individual responsibilities within the teams and an improvement in engagement among less active members.