AcouWrite: Acoustic-Based Handwriting Recognition on Smartphones
Qiuyang Zeng, Fan Li, Zhiyuan Zhao, Youqi Li, Yu Wang · IEEE Transactions on Mobile Computing · 2024
Off-screen handwriting recognitionenriches the handwriting interaction paradigm for mobile devices. However, the existing approaches are only applicable to the specific environment and equipment conditions. In this paper, we proposeAcouWrite, a general, scalable and real-time handwriting recognition system based on active acoustic sensing. In detail, AcouWrite relies onactive acoustic sensingusing only a pair of microphones and speakers on the smartphone to capture real-time handwriting input. Particularly, we extract theshort-time dCIR (st-dCIR)to monitor the changes in the acoustic transmission channel resulting from finger movement. Technically, we use aCNN-GRUclassifier to complete the recognition task in AcouWrite. Moreover, we use data augmentation and spelling error correction methods to improve AcouWrite's robustness. To improve the generalization of our AcouWrite for new characters, we incorporate the transfer learning module into our AcouWrite. In various real-world environments, experiments demonstrate that AcouWrite achieves a mean recognition accuracy of 97.62%, a word accuracy (WA) of 96.4% and a character error rate (CER) of 1.5% for 100 common words, and an average response time of 94 milliseconds.