ExGSense
Chen Chen, Ke Sun, Xinyu Zhang · 2021
Immersive face-to-virtual-face telecommunication is one unique use case for virtual reality (VR) technologies. Existing camera-based telephony systems cannot be used for such immersive VR video chat, due to the physical occlusions of head-mounted displays (HMDs) and/or unwieldy positioning of cameras. To address these, we present ExGSense, a new VR input modality that can sense and reconstruct both upper and lower facial gestures, by only using lightweight biopotential sensors embedded within the HMDs. We optimize the sensor arrangement based on facial anatomy and employ a multiview classification pipeline to exploit the multiple dimensions of signal features. We thus enable ExGSense to detect whole facial gestures by using a sparse set of biopotential transducers. We prototyped ExGSense and evaluated its performance with 42 facial gestures and across different users. We showed a 93% accuracy for user-specific evaluation, and 77% accuracy for user-independent evaluation with low calibration overhead. We believe ExGSense constitutes a promising input modality for immersive VR interactions.