Teach-Me DNA: an Interactive Course Using Voice Output in an Augmented Reality System
Mouna Kenoui, Mohamed Ait Mehdi · 2020
In this paper, we present an interactive system partly based on voice output in an augmented reality environment. Using a chatbot technology, this system allows the user to engage bidirectional communication with a conversational agent. The present system builds upon our previous work in which the user interacts with 3D virtual objects via voice commands. We describe, in the current document, how we integrate speech output using the Text to Speech Service (TTS API) available on the IBM Watson platform, the goal being to obtain an even more interactive system. We also employ Speech Synthesis Markup Language (SSML) to control some features of the natural-sounding speech thus produced. Moreover, we developed Teach-Me DNA, an interactive application that gives the user (pupil, student) the opportunity to learn and/or revise the DNA molecule's basics as a part of a biological course. On one hand, and via voice inputs the user is simply able to perform 3D selections and 3D manipulations of the molecule and its several components in order to learn about their 3D features and properties, this first part dealing with the vocal entries was fully documented in a previous work. On another hand, TeachMe DNA is here enriched by producing a natural speech when reacting to the user's requests. In fact, an agent's voice is used to respond in real time to student's questions. Definitions and/or explanations are thus given in a very natural way which might immerse more significantly the student in the proposed learning environment based on augmented reality technology.