Dynamic Hand Gesture Recognition using Doppler Sonar and Deep Learning
Chiao-Shing Lin, Mohammed Yunus Abdul Gaffar, Jarryd Son, Simon Lucas Winberg · 2021 International Conference on Electrical, Communication, and Computer Engineering (ICECCE) · 2021
This paper explores the use of deep learning to recognise dynamic hand gestures using a sonar system. The study begins by designing and building a Doppler sonar system. The sonar system designed for this study was a multi-static system containing one transmitter and three receivers. The sonar system can measure the Doppler frequency shifts caused by dynamic hand gestures. Since the system uses three receivers, three different Doppler frequency channels are measured. Three additional differential frequency channels are formed by computing the differences between the frequency of each of the receivers. These six channels are used as inputs to the deep learning models. Two different deep learning algorithms are explored to classify hand gestures: a Long Short-Term Memory (LSTM) network and a Convolutional Neural Network (CNN). Six basic hand gestures, two in each x- y- and z-axis, and two rotational hand gestures are recorded using both left and right hand at different distances. Ten-Fold cross-validation was used to evaluate the networks' performance and classification accuracy. The CNN was able to classify all the gestures with an accuracy of at least 98%. The LSTM was able to classify the six basic gestures with an accuracy of at least 98% but with the addition of the two rotational gestures, the accuracy drops to 41%. This result is acceptable since the basic gestures are more commonly used gestures than rotational gestures.