Multimodal dialog management system for collecting patient values and experiences: The HosmartAI perspective
Izidor Mlakar, Matej Rojc, Bojan Musil · AIP conference proceedings · 2024
This paper outlines the development of a multimodal dialog management system for collecting patient reported data and experiences exploiting the symmetric model of interaction.The system targets to improve the quality and sustainability of the collection of patient reports (i.e.patient reported experiences and patient reported outcomes) by enabling the natural channels of interaction, i.e., speech and gestures, on both input and output.The system consists of an Automated Speech Recognition (ASR) to collect information from patient's speech, a Chatbot to understand and interpret the collected information, plan actions and generate natural responses and a Speech Synthesis (TTS), and an Embodied Conversational Agent (ECA) or socially assistive humanoid robot (SAHR), to represent the information using all natural conversational modalities.