Attentive Sequence-to-Sequence Modeling of Stroke Gestures Articulation Performance
Lokesh Kumar T, Luis A. Leiva · IEEE Transactions on Human-Machine Systems · 2021
Production time of stroke gestures is a fundamental measure of user performance with graphical user interfaces. However, production time represents anoverallquantification of the user's gesture articulation process and therefore provides an incomplete picture of such process. Moreover, previous approaches assumed stroke gestures assynchronouspoint sequences, when most gesture-driven applications have to deal withasynchronouspoint sequences. Furthermore, deep generative models of human handwriting ignore the temporal information, thereby missing a key component of the user's gesture articulation process. To solve these issues, we introduceDitto, a sequence-to-sequence deep learning model that estimates thevelocity profileof any stroke gesture using spatial information only, providing thus a fine-grained estimation of themoment-by-momentbehavior of the user's articulation performance. We show that this unique capability makesDittoremarkably accurate while handling gestures of any type: unistrokes, multistrokes, and multitouch gestures. Our model, code, and associated web application are available as open source software.