DWES: A Dynamic Weighted Evaluation System for Scratch based on Computational Thinking
Xiaolin Chai, Yan Sun, Hong Luo, Mohsen Guizani · IEEE Transactions on Emerging Topics in Computing · 2021
Scratch is a programming environment, which is widely adopted as the first language to enter the programming world. Since Scratch projects are rich in multimedia resources, it can be time-consuming to judge them by hand, requiring manual execution with the understanding of each project. Auto-judgment tools come into being, they can rapidly evaluate the project in some way. Unfortunately, the existing tools are too rigid and they ignore the diversity of Scratch projects. To address this issue, we propose a new dynamic weighted evaluation system (DWES). First, we define a new computational thinking (CT) evaluation criteria from eight aspects. Second, based on the proposed criteria, we present an analysis tool that automatically assesses the CT skills of the learners’ projects. Third, considering the characteristics of projects, we dynamically adjust the evaluation results according to types, so that a single standard is no longer applied roughly to all. This process is data-driven, we consider the CT performance and scripts of projects. We prove the rationality of the evaluation criteria from the aspect of program complexity. Through the correlation analysis between DWES CT scores and experts’ ratings, we find that compared with Dr. Scratch, the correlation coefficient has increased.