Your Actions Talk: Automated Sociometric Analysis Using Kinesics in Human Activities
Cheyu Lin, Maral Doctorarastoo, Katherine A. Flanigan · 2024
Cyber-physical-social infrastructure systems (CPSIS) are an extension of cyber-physical systems (CPS). In addition to sensing, measuring, interpreting, and optimizing physical attributes of the built environment to improve infrastructure performance, CPSIS also takes into account human-centered---or social---objectives often overlooked by CPS. Although this paradigm shift aims to incorporate the social system supported by infrastructure into CPS, there is still a gap in measuring social objectives in line with the guiding principle of CPSIS. Specifically, the integration of sensing technologies and computation for assessing these social objectives remains largely unaddressed. As a salient example, sociometric tests used ubiquitously to capture the social structure and sociability embedded within a group of individuals still relies on subjects manually answering questionnaires to derive social connectivity. This data collection scheme is, among other things, subject to attribution bias, inefficient, and laborious. Here, reliance on manually-sourced data to inform sociometric tests falls short in leveraging the sensing and automation capabilities inherent in CPSIS. To overcome these challenges, we propose a human activity recognition (HAR) dataset and framework that can help to automate the procedure of sociometric assessment. The design of the dataset takes into account the limitations that hinders the development of the automated sociometric examination in state-of-the-art HAR techniques. The framework adopts a multidisciplinary approach, drawing upon HAR, kinesics, and sociology to efficiently distill the interpersonal relationships within social systems and provide a qualitative and quantitative interpretation of sociability for modeling and optimization in the context of CPSIS.