Analysis of Online Robotics Challenge Submissions - Fundamental
Sara Willner-Giwerc, Miela Efraim, Chris Rogers · 2021 ASEE Virtual Annual Conference Content Access Proceedings · 2021
Abstract Our goal in this research project was to see what we could conclude about student creativity and learning from looking at student final artifacts. Typically, researchers examine the entire learning journey of students as they complete an engineering design challenge, but we wanted to see how much we could measure and understand about student learning in a situation when we were not able to capture these journeys. This becomes particularly relevant when thinking about asynchronous, online robotics challenges, where often the students’ problem solving processes are not captured in the submissions. We chose the library of student inventions submitted to Dr. E’s Challenges (DrEsChallenges.com) to evaluate because it was an existing, well-defined, public gallery of student work. Professor Ethan (“Dr. E”) Danahy from the Tufts Center for Engineering Education and Outreach (CEEO) released a robotics challenge for students to solve using LEGO Robotics at regular intervals from 2014–2019. The challenge series collected almost 1000 submissions (in the form of images, videos, and short written descriptions) from 57 different countries around the world. These challenges were designed to provide community members (students, teachers, parents, etc.) with an opportunity to create and share solutions to asynchronous online robotics challenges. We conducted a quantitative and qualitative analysis of these submissions in an attempt to discern meaningful indicators of different types of student learning. We were interested in solution diversity (how many different types of submissions were received for each challenge), functionality of solution, complexity of build and code, quality of documentation, use of non-LEGO materials, and variance from given examples. We found that in challenges where examples were given, over 40% of submissions on average for that challenge were a replica of one of the examples provided. Students that instead invented their own solutions tended to have more complex codes and builds. Furthermore, when students invented their own solutions, the length and quality of their documentation increased. Likewise, the 42.6% of submissions that included non-LEGO materials into their build tended to write longer and more descriptive reports to accompany their images and videos. This analysis technique and results have the potential to help online robotics challenge coordinators and creators think critically about the types of submissions they are receiving and the ways that they can improve challenge prompts, supports, and scaffolds for future challenges to elicit engineering thinking.