Processing RGB-D Data from a 3D Camera using Object Detection and Written Character Recognition in Convolutional Neural Networks for Virtual Finger Writing
Ezekiel G. Del Rosario, Carlo P. Nadora, Renee Lou G. Trinidad, Mary Jane C. Samonte, Madhavi Devaraj, Joel C. De Goma · 2020
This study presents an approach to Virtual Finger Writing by treating it as an object detection problem. A Convolutional Neural Network using the Tiny YOLOv3 Architecture was used in this study to enable real-time hand detection. A Kinect camera is used to capture the depth data within the bounding box of the hand region to detect the nearest point in that area which is assumed to be the writing finger. The goal of this study is to present an approach for real-time virtual finger-writing in uncontrolled environments as a possible input method for future hologram or virtual reality systems. After testing and validation of trained model, the study were able to achieve a vast improvement in speed and accuracy results with the application of modern object detection.