FTHAR: A Dual-Modal Deep Learning Model with Feature Engineering for Human Activity Recognition on Wearable Devices
Shuo Xiao, Linhao Cai, Chaogang Tang · 2025
In the application of the Internet of Things, wearable device human activity recognition enables real-time monitoring via sensor data. Previous methods predominantly model 1D sensor sequences directly, with some 2D spatial modeling methods suffering from interpretability limitations. Based on the observation of periodicity in sensor sequences, to address the representational limitations of 1D sequences, we propose a feature extraction method: first, apply frequency domain filtering to decouple frequency bands, thereby reducing interference through band separation while retaining key frequencies and suppressing noise; second, transform 1D sequences into a set of 2D spatial tensors by mining the periodic features of frequency bands, and embedding the periodic characteristics of the sequences into the 2D tensors. On this basis, we construct a dual-modal deep learning model FTHAR for the time and frequency domains, with the two branches complementing each other in representational capabilities. The frequency branch leverages periodic patterns across bands for 2D spatial modeling. Aiming at the differences between dynamic and static activities, the frequency branch first uses a neural network to pre-classify dynamic and static activities, then constructs separate neural classifiers in the frequency domain for accurate recognition. The time branch performs direct 1D temporal modeling on raw sensor sequences. To address prior limitations in capturing cross-sensor correlations, we use an Individual-Merge Convolutional Network: Based on a divide-and-conquer strategy, each sensor uses an Individual Network to extract features independently, avoiding inter-sensor interference; then fuse multi-sensor features via a Merge Network to capture cross-sensor correlations. Code is available at this repository: https://gitee.com/cailinhao/idea1/