Automatic Assessment of Exercises for Algorithms and Data Structures - a Case Study with TRAKLA2
Mikko‐Jussi Laakso, Tapio Salakoski, Ari Korhonen, Lauri Malmi · 2004
AbstractThis paper presents the results of the case study introducing TRAKLA2 system inthe course of data structures and algorithms at the University of Turku in 2004. Wecompared students’ learning results with the results of the previous year. The numericalcourse results were clearly better than in 2003 when only pen-and-paper type exercisesin classrooms were used. In addition, a survey was made with over 100 students onthe changes in their attitudes towards web-based learning environments while gettingacquainted with a wholly new system providing them automatic feedback and the optionto resubmit their solutions. Our results show that such an on-line learning environmentconsiderably increases positive attitudes towards web-based learning, and according tostudents’ self-evaluations, the best learning results are achieved by combining traditionalteaching and www-based learning. 1 Introduction Automatic assessment (AA) tools for CS courses are being developed and gaining acceptancemore and more widely at university level education. The survey of the ITiCSE working group“How shall we assess this” in 2003 indicated clearly that the experience of using AA toolscorrelates with a positive attitude towards applying such methods more widely, also whenassessing higher order skills (Carter et al., 2003). The field where AA is most widely used isassessing programming exercises (e.g. Higgins et al. (2002); Luck and Joy (1999); Saikkonenet al. (2001); Vihtonen and Ageenko (2002)). Other applications include grading algorithmexercises (Bridgeman et al., 2000; Hyvonen and Malmi, 1993; Korhonen and Malmi, 2000)and analyzing object-oriented designs and flowcharts (Higgins et al., 2002).In this paper, we report the experiences on using the TRAKLA2 system for assessingalgorithm exercises in which students simulate working of algorithms on a conceptual level.TRAKLA2 by Malmi et al. (2004); Korhonen et al. (2003) is a visual algorithm simulationexercise system that has been developed at Helsinki University of Technology (HUT). Studentssolve the exercises using graphical manipulation of conceptual visualizations of data structureson the screen. The system provides automatic formative and summative feedback on theirwork, and allows for resubmitting the solutions.TRAKLA2 exercises were used for the first time in the basic data structures and algorithmscourses at HUT in spring 2003. The system was used in parallel with the old TRAKLA systemso that in total 14 TRAKLA2 exercises and 24 TRAKLA exercises were used in two courses