Development of a sign language gesture recognition system based on "RealSense"
Siya Lu, Dahai Ding, Ziqi Wang, W.T. Kuang, Weifan Liu, Xiaoyu Wei, Ran Hu · 2023
With the development of today's society, it is still difficult for deaf mutes and normal people to achieve barrier free communication, and even the communication barriers between them will cause serious contradictions, leading to a strong sense of social belonging that deaf mutes often do not have. If we can use advanced science and technology to achieve rapid and accurate translation of sign language, we will solve the communication barriers between deaf and normal people to a certain extent. Therefore, we will develop a sign language gesture recognition system based on RealSense. So as to realize the recognition of static sign language gestures and dynamic sign language gestures, complete the conversion from body movements to computer algorithms between the deaf and the computer system in the way of sign language, use RealSense camera as the input device to obtain sign language gestures, capture RGB and depth image information, and then use depth information and skin color difference information to segment the captured image information, Then, set up three-dimensional coordinates through RealSense's bone tracking algorithm, so as to obtain the coordinates of hand joints, and deal with them to complete static and dynamic sign language gesture recognition. Combined with software development IDE, set up a sign language gesture recognition system.