Benchmarking of transformer-based architectures for fall detection: a comparative study
Іван Урсул · Technology audit and production reserves · 2025
The object of this research is transformer-oriented deep learning architectures designed for fall detection based on sensor data. One of the main issues identified during the audit of traditional solutions is the excessive computational complexity of standard transformers, which hinders their effective use on resource-constrained devices and in real-time applications. The study involved the use of Temporal Convolutional Transformer, Performer, Multiscale Transformer, LSTM Transformer, Informer, Linformer, and the classical Transformer. Each of these models incorporates advanced mechanisms for attention implementation and processing of both short- and long-term dependencies in input sequences. The Temporal Convolutional Transformer achieved the best results, demonstrating a test accuracy of 99.79% and a peak accuracy of 100% after 50 epochs. This success is attributed to the proposed approach's effective combination of convolutional operations with self-attention, which significantly accelerates the extraction of key features and enables robust handling of short- and long-term temporal dependencies. Convolutional layers help filter out noise from sensor data and reduce computational costs compared to classical transformers. This allows for the deployment of such solutions in real-world edge scenarios without sacrificing fall detection accuracy. Compared to traditional methods, the proposed models offer higher performance and improved resource efficiency – critical factors for implementing real-time fall detection systems. Additionally, the performance of the aforementioned models was evaluated under various operating conditions, including scenarios with low bandwidth and limited energy efficiency. The results confirm that optimized transformer architectures successfully solve the fall detection task while remaining efficient for portable and embedded systems with constrained memory.