SonicOperator: Ultrasonic gesture recognition with deep neural network on mobiles
Xingyu Li, Hongjun Dai, Lizhen Cui, Ya Wang · 2017
With the increasing popularity of smart mobile phones, gesture recognition techniques that enable always available interaction are highly demanded. In this paper, we propose SonicOperator, a system for in-air dynamic gesture recognition based on ultrasound without extra sensors in smart phones, instead of vision-based solutions with camera. Inaudible ultrasonic wave is emitted by speaker and received by microphones, then the frequency-shifted can be modeled with the Doppler effect. Each gesture consists of one or more postures sequentially in chronological order as the duration to perform a gesture is different. Furthermore, recurrent neural network is able to memorize long temporal context for time sequential gesture, to signalizes a soft target in objective function in recurrent neural network training. The experiments demonstrate that the SonicOperator prototype achieves prominent performance through gesture commands in real applications.