Computer Tutor Versus Solving Problems by Hand: A Comparison in Statistics
Colin C. Engebretsen, Paul S. Steif · 2015
Abstract Computer tutor versus solving problems by hand: a comparison in statics While computer-based instruction can be used to automate the process of gradinghomework, sometimes its goal is to provide formative feedback to students whilelearning. Formative feedback is often straightforward to provide when students areperforming simple tasks, such as answering multiple-choice questions. In the case ofproblems that are more complex, with many correct pathways to solution, the task ofenabling the computer to follow student work and provide feedback is much morecomplex.Recently, a computer-based tutoring system was devised that focused on truss problemsin statics. This Truss Tutor has a graphical user interface that enables students to solvesuch problems via method of joints or method of sections in a relatively free form way.Furthermore, by intervening at judiciously chosen points that prevent new parts of asolution from building upon errors, Truss Tutor can give feedback to students regardlessof the solution path they take. This is made possible by a cognitive model for solvingtrusses, in particular a set of algorithms for recognizing, for any current correct state,whether any forward step is correct. When an error is detected, feedback is offered thatenables the user to correct that error and continue with the solution. Students in classes ata number of institutions have completed homework assignments using truss Tutor. Byapplying analysis tools used in intelligent tutoring systems, it has been shown that, formany of the sub-skills necessary for solving truss problems, students commit fewer errorswith practice. So, while students using Truss Tutor improve in their use of the tutor itself,it is unknown how using the tutor compares with completing homework problems in thetraditional manner with paper and pencil.In this paper, we describe a study conducted to compare performance on examinations ofstudents who used Truss Tutor to complete their homework assignment with studentswho solved truss problems by hand. The study took place at an institution in which over600 were enrolled in statics in the current semester. 600 students take statics at the sametime. There are approximately 25 students in each section that are run independently,with some instructors teaching multiple sections. All students in statics take a commonexamination. To provide a sound comparison between computer tutor and pencil andpaper, instructors who taught two or more sections were recruited to participate in thestudy. Each such instructor had one section that used Truss Tutor and one section thatsolved the same truss problems for homework by hand. Sections were also balanced withrespect to time of day and whether the experimental or the control group received lecturefirst. The results of this study, comprising nearly 250 students, currently ongoing, will bereported upon. Development of a cognitive tutor for learning truss analysis Statics poses many conceptual challenges to students and offers exposure to realisticsystems and to a style of analysis important throughout engineering: subsystem isolation.The ability to solve problems is a principal goal of statics; yet students traditionallyreceive the least contact with instructors as they practice solving problems. With thecomputer, one may be able to construct environments that offer instruction and feedbackto students while learning, without the presence of an instructor. But there is a trade off:between how much freedom students have to create solutions to problems and the abilityof the computer to ascertain, judge, and give feedback on what students have done. Onthe one hand, one could give students a blank piece of paper (or a computer tablet) andask them to conduct their analysis; but it would be quite challenging to interpret whatstudents are drawing and respond to it. On the other hand, we could ask students a seriesof multiple-choice questions, with carefully chosen answers; while the responses areinterpretable, they give no indication if students could independently solve a full problemon their own. We refer to this as the latitude-interpretation trade-off.The analysis of trusses via method of joints and method of sections is a topic that is ripefor effective computer-assisted problem solving: students can have reasonably broadlatitude to solve truss problems, while the computer can track students' work in detail andprovide feedback. Truss analysis presents this opportunity because the forms of solutionsare well structured and the common range of student errors can be identified.In this paper, we describe the development of a cognitive truss tutor. The tutor is deemeda cognitive tutor in the sense that there is an underlying cognitive model for the set ofskills or knowledge components needed to solve trusses, as well as the common incorrectactions typical of novice learners. We show how the errors typically committed bystudents in solving truss problems are also allowed by the tutor. We explain how thetutor's design imposes modest constraints on user actions relative to fully free paper-and-pencil solving, but still enable full interpretation of student work. Students have used thetutor in place of written homework in regular statics courses, and data on this usage hasbeen collected. Results from initial analysis suggest that students commit fewer errors asthey use the tutor.