Neural keypoint detection for visual gestures on micro-controllers
Danilo Pietro Pau, Davide Denaro, Marco Lattuada, Mahdi Mseddi · 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · 2022
The human pose estimation, particularly focused on the hands, is an important topic for the metaverse’s researchers. Deep learning has increased the accuracy of keypoints detection applied to deformable objects, with respect to hand-crafted methods. Human hand pose recognition on color images has encouraged new application cases. This paper is focused on the visual hand keypoints estimation estimation in real time deployable on memory and computationally constrained embedded devices. In particular, implementing such technology on microcontrollers is one of the most challenging tasks since the model must be tiny enough to be deployable in less than one megabyte memory. while ensuring adequate accuracy. In this paper, the approach was based on image regression to achieve a proper key points approximation accuracy within the required memory and computation assets of the micro-controller. The quality of projection, complexity and reliability of the proposed solution were validated through comparative analysis of several hyper-parameters combinations, trained ad-hoc and quality measurements. As result, a modified memory optimized version of MobileNetV2 has been developed which required 0.73 MBytes of Flash memory and 0.385 MBytes of RAM memory. The percentage of detected joints achieved was equal to 77.7%. The inference time was, for example, 398 ms at 480MHz on STM32H7 featuring an ARM Cortex M7 instruction set.