The Effect of User-Defined Mid-Air Gestures Elicited by On-Screen Visual Properties on Human Biomechanics, Behaviour, and Perception
Jinghua Huang, Mingyan Wang, Lujin Mao, Ruobiao Wang, Dongliang Zhang, Mengyao Qi, Miaomiao Ke · International Journal of Human-Computer Interaction · 2025
Evidence shows that visual properties of on-screen objects determine their function and operation, thus influencing gesture elicitation. Therefore, this two-stage experimental study investigated how user-defined mid-air gestures elicited by on-screen visual properties affect human biomechanics, behaviour, and perception. Results showed that, psychologically, for the command Next, swiping with the visual flow provided benefits over swiping perpendicular to it; for Remove, swiping perpendicular to the visual flow offered advantages compared to swiping with it. Behaviourally, for Next, swiping left generated shorter response times than swiping up with a stimulus possessing a horizontal visual flow. Physiologically, for Next, swiping up induced higher forearm muscular load than swiping left under the vertical visual flow condition; for View, tapping twice resulted in less finger muscular load under the multiple entities condition than the one entity condition. These findings could provide a scientific basis for guiding appropriate mid-air gesture design considering on-screen visual properties.