A Survey of Deep Learning Approaches for Hand Tracking and Gesture Recognition
Luisinho Carla, João Silva, Maria Pinto, Ricardo Costa, Sofia Almeida, Hugo Martins · 2025
Hand recognition and tracking has become a central problem in computer vision, driven by its vast potential in applications such as virtual and augmented reality, human-computer interaction, robotics, and healthcare. Recent advancements in deep learning have dramatically improved the accuracy and robustness of hand pose estimation, enabling systems to perform complex recognition and tracking tasks under challenging conditions. This survey provides a comprehensive overview of the field, covering key problem formulations, deep learning architectures, evaluation metrics, benchmark datasets, applications, and future directions. We systematically present the mathematical foundations underlying 2D and 3D hand pose estimation, review prominent network designs including convolutional, graph-based, and transformer-based approaches, and discuss the challenges of generalization, data efficiency, and real-time inference. Special attention is given to the integration of physical and anatomical priors, the role of self-supervised learning, and emerging trends such as hand-object interaction modeling. By synthesizing insights from the latest literature, this work aims to serve as a resource for researchers and practitioners seeking to advance the state of hand recognition and tracking, and to outline promising avenues for future exploration.