Measuring Student Computational Thinking in Engineering and Mathematics: Development and Validation of a Non-programming Assessment
Timothy Duckett, Gale A. Mentzer · 2020 ASEE Virtual Annual Conference Content Access Proceedings · 2020
Abstract With the emphasis on the development of computational thinking (CT) skills comes the challenge of accurately measuring CT. Because of its close association with computer science, CT is often measured using programming tools (such as Scratch, Zoombinis, gaming, or simulation-based situations) on a computer (Shute, Sun, & Asbell-Clarke, 2017). CT skills, however, go well beyond programming and should be measurable as a skill that one can implement in other problem-solving situations (Berland & Wilensky, 2015). The majority of CT measures that do not use technology and programming as the medium for measurement are project-specific, examine attitudes towards CT, use a longitudinal approach by examining a project-based process (Shute, Sun, & Asbell-Clarke), or do not examine the transfer of CT to situations other than computer programming (Bers, et al., 2014). This presentation shares the development and validation of a student CT test that can be completed as an online or paper and pencil survey. While developed as part of an NSF STEM +C project designed to improve mathematics and CT ability and interest through learning how to program self-driving model cars, the CT assessment was created as a generic test of CT based upon mathematics because it was administered to both the intervention and control students in high school mathematics courses. In addition, it was the goal to create an assessment that could be completed in less than 30 minutes yet provide a valid measure of student CT. Our assessment is based upon the ComputationalThinkers.com framework (which in turn is based upon Wing's seminal article (2006)): CT requires students to take a complex problem and break it down into a series of small, more manageable problems (decomposition). The smaller problems can be looked at individually, considering how similar problems have been solved previously (pattern recognition) and focusing only on the important details, while ignoring irrelevant information (abstraction). Next, simple steps or rules to solve each smaller problem can be designed (algorithms). The assessment has a total of 15 items. The first eight items are multiple choice asking students about preferred problem-solving process. The remaining seven items are open-ended and ask students to elaborate on the steps they would take to solve problems like finding the fastest route from a bus stop to the library (road map is included) and finding the area of an irregular polygon (students are asked to list the steps or process, not solve the problems). This presentation examines the reliability and validity of the test and explores whether there is a change in CT skills between pre and post participation. It also explores whether there are differences not only between the intervention and control groups but looks at student demographics like age, gender, race, and education level. Berland, M. & Wilensky, U. J Sci Educ Technol (2015) 24: 628. Bers,M., Flannery,L., Kazakoff,E., Sullivan,A., 2014.Computational thinking and tinkering: .Exploration of an early childhood robotics curriculum. Computers & Education 72, 145–157. Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142-158.