HackWrt: Network Traffic-Based Eavesdropping of Handwriting
Aaron T. Kinfe, Chijung Jung, Kai Biao Lin, Marshall Clyburn, Fnu Suya · 2023
The rapidly expanding world of IoT continues to produce new products that are a combination of familiar tools and concepts with new technology, such as the smart watch and the smart home. The smartpen is one such product. Smartpens are capable of recording and transmitting user content to a connected device. This allows users to effortlessly digitize their handwritten content, but this also means that users can—through the use of smart pens—inadvertently expose sensitive, handwritten information as it passes from the pen to the host device. For instance, a doctor could use a smart pen to quickly note down patient observations. More simply, a user could use one to record private information they would rather keep from prying eyes. This being the case, it is important that we evaluate the security around using a smart pen. In this paper, we present HackWrt, a method of recreating handwritten characters from raw, possibly encrypted traffic. This paper details the method by which HackWrt reconstructs handwritten text. We also evaluate its performance and accuracy on a dataset. The promising results we obtained show that this is a feasible method for exfiltrating data.