Enhancing STEM Education using Augmented Reality and Machine Learning
Ivan Jie Xiong Ang, King Hann Lim · 2019
Learning Science, Technology, Engineering and Mathematics (STEM) in the 21st century has been evolved from the conventional textbook to the interactive platform using electronic devices. This paper presents the implementation of a mobile application system, named AUREL (Augmented Reality Learning) in enhancing the learning experience by projecting Augmented Reality (AR) objects onto 2D images. This AR visualization is used to improve the understanding of STEM subjects and increases the enthusiasm of students towards STEM subjects. In this implementation, Google's Cloud Tensor Processing Units (TPUs) are used to train specific datasets alongside Cloud Vision API to detect a wide range of objects. ML Kit for Firebase is used to host the custom TensorFlow Lite models for specific use cases for better accuracy. On the other hand, Google Cloud Platform (GCP) is used to harvest STEM data, manage STEM 3D information and data processing. Subsequently, the processed information will be displayed in AR in the mobile application using ARCore's Sceneform SDK. The application of AUREL could be extended to all science subjects so that students can learn using an interactive platform.