Predicting Personality Traits from Hand-Tracking and Typing Behavior in Extended Reality under the Presence and Absence of Haptic Feedback
Jonathan Liebers, Felix Bernardi, Alia Saad, Lukas Mecke, Uwe Gruenefeld, Florian Alt, Stefan Schneegaß · 2025
With the proliferation of extended realities, it becomes increasingly important to create applications that adapt themselves to the user, which enhances the user experience. One source that allows for adaptation is users’ behavior, which is implicitly captured on XR devices, such as their hand and finger movements during natural interactions. This data can be used to predict a user’s personality traits, which allows the application to accustom itself to the user’s needs. In this study (N=20), we explore personality prediction from hand-tracking and keystroke data during a typing activity in Augmented Virtuality and Virtual Reality. We manipulate the haptic elements, i.e., whether users type on a physical or virtual keyboard, and capture data from participants on two different days. We find a best-performing model with an R² of 0.4456, with the error source stemming from the manifestation of XR, and that the hand-tracking data contributes most of the prediction power.