Over-The-Shoulder Training Between Redundant Wearable Sensors for Unified Gesture Interactions
Md Aashikur Rahman Azim, Adil Rahman, Seongkook Heo · 2022
Wearable computers are now prevalent, and it is not uncommon to see people wearing multiple wearable devices. These wearable devices are often equipped with sensors to detect the user’s interactions and context. As more devices are worn on the user’s body, there is an increasing redundancy between the sensors. For example, swiping gestures on a headphone are detected by its touch sensor, but the movement it caused can also be measured by the sensors in a smartwatch or smart rings. We present a new mechanism to train a gesture recognition model using redundant sensor data so that measurements from other sensors can be used to detect gestures performed on another device even if the device is missing. Our preliminary study with 13 participants revealed that a unified gesture recognition model for touch gestures achieved accuracy for 25 gestures (5 gestures × 5 scenarios) where gestures were trained by leveraging the available sensors.