AudioWrite: A Handwriting Recognition System Using Acoustic Signals
Lin Wang, Junbao Zhang, Yue Li, Haoyu Wang · 2023
In this paper, we propose AudioWrite, an acoustic-based handwriting recognition system. It utilizes the built-in microphone to capture sounds produced by the finger sliding on the surface of the object to recognize handwritten characters. We first design a segmentation algorithm to detect handwritten sounds in real time and accurately segment character signals. Then we extract time-frequency features from sound signals by performing short-time Fourier transform (STFT). Moreover, in order to obtain sufficient training samples at low cost to build an effective classification model, we adopt Generative Adversarial Network (GAN) to generate synthetic data with only a small set of real-world data. Since mobile devices are usually limited in storage and computing resources, we use the lightweight ShuffleNetV2 model to recognize handwritten characters. We implement AudioWrite as an Android application and conduct experimental evaluations. The results demonstrate that AudioWrite can recognize handwritten characters (10 numbers and 26 uppercase letters) in real time with an overall accuracy of 92.2%.