Hybrid YOLO-InceptionResNetV2 Pipeline for Automated Human Activity Recognition in Controlled Environments
Pranay Mandadapu · 2025
Human activity recognition (HAR) plays a vital role in healthcare monitoring, but manual video annotation remains a time-consuming and error-prone process. This study presents a hybrid deep learning pipeline that automates HAR from controlled environment videos by integrating You Only Look Once (YOLO)-based human detection with Inception-ResNetV2 classification. The system first employs YOLOv8 to isolate human subjects from the surrounding background and then uses separate models for cropped and uncropped frames, selected dynamically based on human detection. Experiments were conducted on a dataset of 18 subjects, each recorded for 12 hours in a metabolic chamber, representing real-world activities such as sitting, standing, walking, and lying. The hybrid approach achieved an overall accuracy of 66.11%, improving by 9.43% compared to the baseline model without YOLO preprocessing. These results show that isolating human subjects improves classification performance (66.11% accuracy, +9.43% over baseline), although overall accuracy remains modest and is influenced by class imbalance. The findings suggest potential for automated activity monitoring in controlled healthcare environments.