Studies in conversational AI : multilingual capabilities, world knowledge, and evaluation strategies
Maxime De Bruyn · 2024
This thesis studies the evolving landscape of conversational AI.The main research objective is to improve the conversational abilities of conversational agents, with a focus on integrating real-time knowledge and expanding multilingual capabilities. Integration of External KnowledgeThe thesis investigates how to incorporate external knowledge into conversational agents without the need for retraining the entire model.This aspect is crucial as it deals with the dynamic nature of information and the need for AI agents to stay updated. Assessment of Inherent World KnowledgeAnother critical problem is how to evaluate the inherent world knowledge that users expect from conversational agents.This involves benchmarking the agents' common sense and broad understanding of the world, which is essential for natural and relevant interactions. Refinement of Agent ResponsesThe research also explores refining the agents' ability to select the most appropriate response from a set of potential replies. Multilingual CapabilitiesThe thesis recognizes the growing need for conversational agents to be proficient in languages other than English.It examines how additional datasets can be used to develop agents capable of operating effectively across a multitude of languages.Evaluation Metrics Finally, the thesis addresses the challenge of evaluating conversational agents.Unlike many machine learning applications where a gold standard or reference exists, conversational agents require metrics that acknowledge the multifaceted and subjective nature of conversations, where multiple valid continuations exist.i ii Overall, the thesis presents a comprehensive approach to enhancing knowledgegrounded conversations, emphasizing better access to external and world knowledge, enhancing non-English language capabilities, and developing more effective evaluation strategies.It synthesizes findings from various interdisciplinary studies and sets a path for future research in the field of conversational AI.