AcousticType: Smartwatch-Enabled Cross-Device Text Entry Method Using Keyboard Acoustics
Ülkü Meteriz Yýldýran, Necip Fazýl Yýldýran, Aziz Mohaisen · CHI Conference on Human Factors in Computing Systems Extended Abstracts · 2022
The integration of various technologies with our daily lives, such as smartwatches, smartphones, smart TVs, etc. has proven beneficial. However, the HCI community is still in search for more intuitive text entry methods for smart devices. In this study, we explore a new text entry method—AcousticType for smart devices where a smartwatch is leveraged to infer what a user is typing on a physical keyboard employing the acoustic signals as a secondary input mechanism. AcousticType employs four modules: Noise Cancelling, Keystroke Detection, Key Identification and Word Correction. Our results show AcousticType recovers up to 98% of the typed text. We also showed that utilizing the noise cancellation, AcousticType is robust to the changes in the environment, which further boosts the practicality. The findings are promising and call for further investigation into new types of text entry methods that utilize acoustic signals to cope with the usability issues in smart devices.