Optimized NLP Models for Digital Twins in Metaverse
Valeria Lukaj, Alessio Catalfamo, Maria Carolina Fazio, Antonio Celesti, Massimo Villari · 2023
Digital Twins (DTs) in Metaverse face many challenges such as the lack of optimized AI models to allow the interaction between the user and the virtual environment. In this paper, we propose an optimized model for human language processing based on Convolutional Neural Networks (CCNs) and we present an input processing strategy to meet the real-time requirements of smart applications that integrate DTs oriented to speech-based functionalities for user interaction and Metaverse. In our solution, CNNs are applied for the processing and classification of the human voice, while structured data and MFCC coefficients are used to train the neural networks and generate interference in the models. Similarly, the MFCC algorithm is provided to extract the unique characteristics that specify each generated audio file and to reduce the complexity of the neural network model in order to obtain better performance. Starting from an approach to the problem available in the literature, we have optimized a specific CNN model for Natural Language Processing (NLP) in order to increase effective results. The proposed model has demonstrated excellent performance and can be used as a basis for the implementation of software that allows the interaction of DTs with voice commands issued by a user.