SemiGest: Recognizing Hand Gestures via Visible Light Sensing with Fewer Labels
Jifei Zhu, Ziwei Liu, Yimao Sun, Yanbing Yang · 2023
Human-machine interaction (HMI) is much important in factories, and most HMI ways are contact which arise safety and health issues. To avoid such problems, contactless HMI ways such as in-air hand gesture recognition (HGR) via Wi-Fi or radar are widely studied by both academia and industry. However, these RF-based methods are not very appropriate for the industry because of the electromagnetic interference. As for visible light sensing, it is free of electromagnetic radiation and can reuse the existing devices, e.g., lamps on machines, hence utilizing visible light to realize HGR is a good solution for HMI in factories. The current visible-light-enabled HGR (VL-HGR) methods using deep learning algorithms are all supervised, which increases the cost of manual labeling and further hinders the industrial applications of VL-HGR. To this end, we propose SemiGest, a semi-supervised learning (SSL) method for VL-HGR, to facilitate the applications of VL-HGR in industry. The system prototype is built on a table lamp to mimic the lamp on a machine emitting lights at four distinct carrier frequencies, and the lights reflected by hands are collected by a receiver. SemiGest utilizes the variation and correlation of the lights to realize HGR with an SSL algorithm using only a small amount of labeled data and lots of unlabeled data. Furthermore, the SSL algorithm is designed not only for the visible light data but also can be generalized to other time-series data in the industry. To confirm the effectiveness and robustness of SemiGest, we perform various experiments to show the potential for practical implementation in the industry.