Effects of Automation Explanations on Operator Reliance on a Context-Aware Recommendation System
Chelsea Carrasco · TSpace (University of Toronto) · 2018
This thesis explores the design, requirements definition and development of an Adaptive User Interface (AUI) that provides context-aware action recommendations to operators performing a cybersecurity network monitoring task. This AUI was used to conduct an experiment that investigated whether providing meta-information about how the automation calculated recommendations would impact trust and reliance on those recommendations. Twenty-four participants performed a cybersecurity task under three different interface conditions, once each with real, random and no explanations of how recommendations were calculated. Results did not show any significant effect of the explanations on operators trust of the system, nor on their reliance on the system recommendations. A significant correlation was found between a participant’s propensity to trust (PTT) automation and their trust ratings, and between PTT and their reliance on recommendations. Results appear to indicate that explanations provided on the AUI were not useful enough to impact participants trust as hypothesized.