Detecting Face-Touching Gestures with Smartwatches and Deep Learning Networks
Sakorn Mekruksavanich, Wikanda Phaphan, Anuchit Jitpattanakul · 2024
Everyday actions like scratching one's nose or resting the chin on one's hand may facilitate the spread of germs and diseases. Detecting these gestures holds the potential for innovative health monitoring and disease prevention applications. However, the identification of face-touching poses challenges due to the variety of human gestures and the limitations in accuracy associated with wearable sensors. This study introduces an original deep-learning system to identify face-touching gestures using standard smartwatches' inertial measurement unit sensors. The system on the smartwatch captures and pre-processes multi-channel time-series data from the accelerometer to generate robust input features. We utilize a benchmark, the Face Touching dataset, to evaluate the recognition performance of deep learning networks, including our proposed network. Our approach proposes a hybrid deep residual network architecture tailored for sequence-based gesture classification using signals from the smartwatch sensors. In our experiments, the developed deep learning framework achieves an impressive F1-score of 97.53% in detecting face-touch gestures. The suggested system takes a step forward in advancing the practical applications of face-touch gesture detection to smartwatch-based health sensing and disease prevention.