An ML pipeline for real-time activity detection on low computational power devices for metaverse applications
Amit Kumar, Amanpreet Chander, Ashish Kumar Sahani · 2023
This paper presents our recent work on real-time human activity detection based on the mediapipe pipeline and machine learning algorithms. A single webcam was used to capture the subject images and the data was recorded in a csv file. These CSV files were used to training and testing machine learning (ML) modes. The proposed system can detect human activities including running, jumping, squatting, bending to the left or right, and standing still. An open source framework named Mediapipe has been used to identify body makers which in result used for body pose. We tried hard coded approach and machine learning based approaches. Four models including random forest (RF), K-nearest neighbor (KNN), neural networks (NN) and convolution neural network (CNN) were tested. NN and CNN perfomed best with NN obtaining maximum accuracy of 99.76% and CNN obtaining accuracy of 100% on the test data sets. The proposed solution provides a cheap and affordable solution for home based exergaming ang rehabilitation.