A Virtual Assistant with Real-Time Sign-to-Sign Language Translation for Users with Disabilities
M. Vijay, B. Aditya Vighnesh, Chitturi Santosh, S M Dhanush, Ch. Hemanth · 2024
As technology progresses, the inclusion of sophisticated language models like ChatGPT in virtual assistant systems opens up new avenues for user engagement. This paper describes the creation of a virtual assistant that combines ChatGPT's sophisticated conversational capabilities with the operational characteristics of a typical assistant, such as Google Assistant, while leveraging the ChatGPT API for more functionality. In order to respond with customized solutions, the system uses sentiment analysis via VADER to understand the emotional tone of user inputs. Furthermore, a pre-defined gesture library translates text output into Indian Sign Language (ISL) motions, facilitating seamless communication with sign language users. The goal of fusing emotional intelligence, sign language recognition, and operational control is to produce a more productive and engaging user experience. This article covers the architecture of the system, the integration of natural language processing, and potential applications.