An Attention-Based Multi-Modal Framework for Intelligent Elderly Care Using Activity and Acoustic Data
Raoudha Nouisser, Salma Kammoun Jarraya, Mohamed Hammami · Procedia Computer Science · 2025
Although many elderly monitoring approaches either focus on activity or acoustic analysis, very limited work integrates both modalities within a unified framework for emergency detection. This paper presents a multi-modal method that combines skeleton-based activity with acoustic analysis to enhance the detection of medical emergencies in elderly individuals. A comprehensive pipeline is introduced, encompassing data preprocessing, feature extraction, multi-modal fusion, and emergency classification. Deep neural networks are leveraged to extract meaningful representations from both skeleton and acoustic data, while an advanced attention-based model refines the fusion process by prioritizing the most relevant features. The proposed method is validated through extensive experiments on specialized datasets for geriatric monitoring, demonstrating its effectiveness in accurately identifying abnormal activities and sounds associated with emergencies. Notably, the incorporation of attention mechanisms significantly improves detection reliability. This work contributes to the development of real-time continuous monitoring systems, promoting early intervention and reducing dependence on acute care.