Chat with Smart Conversational Agents
Xiantao Chen, Jiaqi Mi, Menghua Jia, Yajuan Han, Moli Zhou, Tian Bao Wu, Daisong Guan · 2019
There are more and more smart devices equipped with smart conversational agents, which can engage in chat or free conversation with human. However, the human-machine chat is still in the early stage of development, and there is a lack of effective methods to evaluate chat experience. In this study we proposed an approach to evaluate chat experience with smart conversational agents in smart home. We collected evaluation metrics and applied them in user testing, and then optimized the metrics and constructed an evaluation system. We applied the evaluation system to compare chat experience of five different smart conversational agents.