A Data Efficient Vision Transformer for Robust Human Activity Recognition from the Spectrograms of Wearable Sensor Data
Jamie McQuire, Paul Watson, Nick Wright, Hugo Hiden, Michael Catt · 2023
This study introduces the Data Efficient Separable Transformer (DeSepTr) architecture, a novel framework for Human Activity Recognition (HAR) that utilizes a light-weight computer vision model to train a Vision Transformer (ViT) on spectrograms generated from wearable sensor data. The proposed model achieves strong results on several HAR tasks, including surface condition recognition and activity recognition. Compared to the ResNet-18 model, DeSepTr outperforms by 5.9% on out-of-distribution test data accuracy for surface condition recognition. The framework enables ViTs to learn from limited labeled training data and generalize to data from participants outside of the training cohort, potentially leading to the development of activity recognition models that are robust to the wider population. The results suggest that the DeSepTr architecture can overcome limitations related to the heterogeneity of individuals’ behavior patterns and the weak inductive bias of transformer algorithms.