A Two-Stage Methodology Combining Clustering and Predictive Models to Estimate the Levels of Participants’ Empathy and Other Affective States During Virtual Reality Sessions
Emilija Kizhevska, Hristijan Gjoreski, Mitja Luštrek · IEEE Access · 2026
Virtual reality (VR) has been described as the “ultimate empathy machine” due to its ability to immerse users in perspectives beyond their own, enhancing emotional engagement. In this study, 105 participants experienced 360° VR videos portraying actors expressing core emotions: happiness, sadness, anger, and anxiety. Empathy was assessed through self-report questionnaires, alongside other affective states including arousal, valence, and discomfort. Physiological and expressive responses were recorded using multimodal sensor data that captured facial muscle activity, heart rate, and motion dynamics. Extracted features reflecting central tendencies, variability, and distributional patterns were used to cluster participants into distinct groups, revealing inter-individual differences in emotional and empathic engagement. Cluster-specific predictive models, including Random Forest (RF) and deep neural networks (DNN), were then trained to predict state empathy and other affective states by leveraging unique patterns within each cluster, achieving 75 percent balanced accuracy for empathy prediction (RF) and even higher results for other affective states. This study demonstrates a systematic approach for quantifying empathy and affective processes in VR through multimodal sensor data. The methodology highlights how physiological and expressive signals capture meaningful differences in engagement, supporting real-time, personalized prediction. These findings provide a foundation for objective assessment of empathy and other affective states and contribute to the development of immersive VR applications in research, clinical, and educational contexts.