Salat Activity Recognition on Smartphones using Convolutional Network
Syed Hamza Mohiuddin, Tahir Syed, Behraj Khan · 2022
Salat, the penta-quotidian form of worship, is an obligation on all Muslims. Vital to Salat is a set of poses and gestures that must be performed in a specific sequence. It is possible for a person to forget their current state in the sequence and skip some of the steps perhaps because the person is new to worship. In this paper, we develop a smartphone-based solution that guides the person through all gestures and poses in the correct sequence via visual and audio feedback. We formulate the problem as human action recognition using per-pose recognition using convolutional neural networks embedded within a deterministic finite automaton of pose transitions. With this approach that uses video frames instead of volumes, and therefore with a considerably smaller model, we achieve better test accuracy on the benchmark dataset, i.e. 83% vs. 78% of the nearest competitor. We also create a larger and finer-grained proprietary dataset with eight classes, and report a test accuracy of 88%.