PokerFace Mask: Exploring Augmenting Masks with Captions through an Interactive, Mixed-Reality Prototype
Josh Paul Davis · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · 2022
The 2020 COVID-19 pandemic made masks a daily wearable for personal protective equipment as a public health precaution.Traditional mask designs obscure communication by obstructing the face and muffling the voice which can make communication especially difficult for users who are deaf or hard of hearing (DHH).PokerFace uses a commodity smartphone and recycled materials to display a live-stream of a user's mouth and nose on the mask surface.This maintains the safety precautions afforded by the mask, while mitigating the obfuscation of traditional mask designs.To compare PokerFace's ability to facilitate communication with traditional masks, we conducted a user study with 18 participants, who played a collaborative communication game similar to charades.Participants performed better at this collaborative communication task with our prototype than with traditional masks, and even non-DHH users became aware of the importance of lip-reading and facial cues in communication due to study participation.In this paper, we present PokerFace, an interactive, mixed-reality mask which allows for face-worn interaction and expression using a commodity capacitive touch-screen device and cost-effective recyclable materials (Figure 1b, 1c, and 1d).The mask situates a smartphone in a cradle over the user's mouth and nose, similar to a traditional mask.PokerFace uses the display of the device to live-stream a video of the wearer's mouth and nose in real time, as well as present captions of user's speech using a real-time speech-to-text function.Our goal with this prototype was to explore the following research questions:• R1.How can embedding computation into