Exploiting Semantic Search and Object-Oriented Programming to Ease Multimodal Interface Development
Thibaut Septon, Santiago Villarreal-Narvaez, Xavier Devroey, Bruno Dumas · 2024
Multimodal interaction has been adopted across various platforms and devices, with supporting tools enhancing the developer experience in developing (). While traditionally, these tools faced challenges balancing expressiveness and usability, recent progress in tends to mitigate this rule. However, adding multimodal interaction still remains challenging, especially when integrating the voice modality, and s remain to be better integrated into today’s applications. To address these challenges, we introduce a Unity tool-based system named. allows developers to use their knowledge in Object-Oriented Programming to handle the expert knowledge required to create a while allowing end users great flexibility in the natural language they can use to interact. Our contributions are: 1) the presentation of ’s architecture and its inherent concepts along with its open-source implementation, and 2) a successful evaluation of its usability for describing s through the questionnaire with twelve participants.