Ordered Network Analysis in CS Education: Unveiling Patterns of Success and Struggle in Automated Programming Assessment
Andres Felipe Zambrano, Maciej Pankiewicz, Amanda Barany, Ryan S. Baker · 2024
Computer science (CS) education at the university level is often challenging, particularly for students with no prior programming experience. To help scaffold students' CS learning, instructors often utilize systems for automated assessment of programming assignments, where students can individually learn online using automatically generated feedback. However, despite the growing usage of these systems, learning outcomes are often mixed and not all students benefit equally from using these applications. In this study, we utilize Ordered Network Analysis (ONA) to examine data from a system for automated assessment of programming assignments and compare platform activity between novice students (N=110) achieving high (N=43) and low (N=67) scores on the final test of an introductory CS course. We identify and visualize differences in the activity patterns between the groups. High performing novice students tend to request feedback more often, while low performing students more often leave the assignment unsolved after experiencing an unsuccessful attempt. These findings show that Ordered Network Analysis can serve as a useful tool for understanding student behaviors, facilitating the design of targeted interventions that might support learners at key moments in their programming engagement towards task success.