Evaluating 3D Gesture Recognition in UAV-Based Human-Robot Interaction
Paxton C. Albright, Scott M. Martin, Chad G. Rose · IFAC-PapersOnLine · 2025
As the applications for Uninhabited Aerial Vehicles (UAVs) continue to grow, a quick and intuitive method of communication with these systems, independent of intermediate hardware, will be vital. Prior work has shown the feasibility of vision-based gesture recognition as a means of conveying commands to UAVs. However, practical considerations about gesture recognition performance from the dynamic viewpoints of UAVs have not been fully explored. In this work, we establish gesture set design criteria through a simulation-based study on static 2D (planar) and 3D (multi-planar) gesture sets. Data was captured using varying distance, angle, lighting, scene, and character models within the Unity game engine to construct a dataset of 147,420 images. Six instances of YOLOv11 object detection were trained on the data to capture the effects of these factors as image resolution is changed. The results of this study showed that designing gestures to have large and uniquely shaped bounding boxes leads to improvements in performance across a wide range of viewing angles that result from UAV operations.