Assessing user satisfaction with information chatbots: a preliminary investigation
Divyaa Balaji · 2019
Despite the increasing number of service chatbots available today, it has been observed that many often fail to impress (Brandtzaeg & Folstad, 2018) - chatbot developers may thus benefit from a diagnostic measure of user satisfaction. This thesis follows previous work on developing a diagnostic questionnaire that assesses user satisfaction with information chatbots. A pre-experimental phase was undertaken in which the original list of chatbot features obtained from Tariverdiyeva and Borsci (2019) was reviewed by a team of experts and an extended literature review was conducted to ensure that all relevant chatbot features had been identified - the resulting list was used to generate an item pool. Study 1 describes a series of focus groups in which participants discussed the updated list of chatbot features and item pool based on which further refinement took place. Study 2 describes the administration of the item pool to a sample of university students and analyses conducted in order to test the questionnaire’s factor structure. It was found that the data acquired from participants can be captured by four factors - communication quality, interaction quality, perceived privacy and perceived speed. Actions for future studies are discussed in order to arrive at the desired questionnaire.