A New Method to Increase Feedback for Programming Tasks During Automatic Evaluation
Piroska Biró, Tamás Kádek, Márk Kósa, János Pánovics · Acta Polytechnica Hungarica · 2022
The unexpected challenges posed by the pandemic also have transformed university education.Information technology is still the most advantageous field, as IT tools in education are more widespread.We have been using the ProgCont system for automatic evaluation of programming tasks since 2011 at the Faculty of Informatics of the University of Debrecen.The system's responsibilities have expanded over the years, and due to the pandemic, it will have to play a more significant role in self-preparation.Initially, we used the system to evaluate competitive tasks and later examinations.In this period, the feedback was limited to accepting or rejecting the submitted solutions.A submitted solution is accepted if the application produces the appropriate output for the problem's input.Usually, we test the submissions with several inputs (test cases) for each problem.To provide additional information about the reason for rejection, we would like to supplement test cases with comments (annotations) that identify the test cases' unique properties.Our goal is to help identify the subproblems that need improvement in case of a partially correct solution.In our article, we would like to present the potential of this development.We chose a problem that received an impressive number of solutions.We created new test cases for the problem with annotations, and by re-evaluating the submissions, we compared how much extra information students and instructors obtained using the annotations.The presented example proves that this new development direction is necessary for students' self-preparation and increases differentiated education possibilities.