Deep Learning-based Digital Twin for Human Activity Recognition
Jian Dong Su, Zhenlong Liao, Qiankun Mao, Zhengguo Sheng, Alex X. Liu · 2023
With the rapid development of the Internet of Things (IoT) related technologies, the application of digital twins (DT) in industry and healthcare becomes possible. Human activity recognition (HAR) is emerging as a hot research area with great potential in healthcare. Activity recognition systems combined with DT will make it easier to monitor human health conditions to improve the quality of life and happiness with individualized healthcare. In this paper, we design an effective HAR system, called HAR-Net, which uses WiFi time series data collected by sensors to train a deep learning network. Deep learning’s great learning ability is utilized to extract features of various human activities for activity recognition. We built the DT system with Unity, which is combined with the HAR system. In the DT system, real-world physical activities are mapped onto human models. The results of activity prediction can be evaluated in real-time in DT, and warnings can be issued quickly when dangerous activities occur. To make our human activity recognition system more adaptive, we propose a one-shot recognition method based on meta-learning. Specifically, we design a Bi-path basic network that extracts features in the time-domain and frequency-domain, and a meta-learning framework with a classification module and a WiFi metric module. Using datasets from different environments, we conducted various experiments on HAR-Net, and the results proved that our presented method was superior to the baseline network.