Real-time air-writing recognition in motion stream
Buntueng Yana, Takao Onoye · International Workshop on Advanced Image Technology (IWAIT) 2019 · 2019
The main contribution of this work is developing an end-to-end air-writing recognition technique for a real-time application. We assume the user performs the air-writing naturally and intuitively without doing any explicit signal. For avoiding the spotting process, this work considers the segmentation free technique using the LSTM network with CTC loss. The fusion scheme models the writing trajectory with the spatial and temporal features. To extract the writing information from the finger motion, we utilize a window-based technique for segmenting stream data for generating the training features. We deploy two features: the hand position and the path signature, to train the proposed network. For evaluating the performance of the proposed technique, we conduct the experiments the public dataset namely the finger writing. From the result, it confirms the fusion scheme can improve the recognition accuracy. The appropriate size of the sliding window for the proposed structure is 0.25 second while the skip size equals 83 milliseconds. The proposed network can recognize the air-writing word 75.81% without the language model. When considering the processing time of the recognition technique, the air-writing could predict the written word within 6.37 milliseconds. It confirms the proposed algorithm can deploy for a real-time application.