Evaluation of the Naturalness of Chatbot Applications
Ayah Atiyah, Shaidah Jusoh, Firas Alghanim · 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT) · 2019
Chatbots are intelligent machine-to-human conversation systems. The main benefit of developing and deploying a chatbot in any business domain is that it can behave as a personal assistant. A chatbot is expected to be close in efficiency of interaction to a human personnel in responding to customers questions. The aim of this study is to evaluate the performance and naturalness of interaction of chatbot applications. The performance of a personal chatbot has been compared to a human personal in order to evaluate the naturalness. The evaluation was made by conducting an experiment on two expert users using cognitive walkthrough method. Findings of the experiment suggests that the human-machine chatbot is able to be close to the human performance in terms of interaction, although in some minor cases, the human-human chat applications perform better.