On the usage of smart devices to augment the user interaction with multimedia applications
Alan Ferrari, Vanni Galli, Daniele Puccinelli, Silvia Giordano · 2017
Wearable devices have recently gained a foothold in the market with the uptake of smartwatches. The strong tie between a smartwatch and its owner, the highly predictable position of a smartwatch on the body, and its internal sensors are enabling a wide array of applications that leverage the user context. In this paper we focus on a gesture recognition system to augment the user interaction with multimedia applications. We define a set of seven gestures that are relevant across several applications and we collect an extensive dataset with two smartwatches (the Motorola Moto360 and Apple's Watch). We use Long Short Term Memory neural networks for gesture recognition based on sensor data from both smartwatches. We provide an extensive evaluation of the classification accuracy of the system and provide a sensitivity analysis to find the Long Short Term Memory configuration that maximizes the classification accuracy. We also show the extent to which Long Short Term Memory neural networks outperform traditional machine learning approaches. We also illustrate an application we built for the Android and iOS platforms that allows developers to easily integrate the gesture recognition in their own systems. We conclude the paper with a description of use cases to underscore the potential impact of our contribution.