Use of Computer Vision Based Hand Tracking in Educational Environments
V. Kriznar, Mirjam Leskovšek, Borut Batagelj · 2021
Computer vision and it's use is starting to represent a major part of our daily life. Analysis of captured pictures and the use of the information that they contain can support specific tasks in an increasing number of applications. In this paper we present two out of the box solutions for making teamwork-based education more interesting by using wireless plugins. For this reason we used two different systems for detecting palms and fingers of user's hands. We developed two different applications to test each of the two out of the box solutions. The first application supports teacher-student cooperation. Students on remote computers can draw various shapes and lines fully via hand tracking and the shapes are then simultaneously redrawn on the teacher's or student's computer. It uses Google's MediaPipe library for hand tracking and Node.js for server communication. The second application is a simple video game aimed primarily at preschool children. By playing the game, children learn basic shapes, colors and logical thinking. The application uses the handtrack.js library for hand-tracking and HTML/CSS with JavaScript for the base application. Both applications demonstrate how computer vision can be easily integrated into remote educational user interfaces. We hope that this will stimulate other developers of educational applications to integrate into their application also computer vision based interaction.