eXplainable AI Interfaces With (and for) Expert Operators: A Participatory Design Approach
Negin Hashmati, Hugo Wärnberg, Emmanuel Brorsson, Mohammad Obaid · 2024
This paper explores the integration of user-centered participatory design (PD) methodologies to develop feedback solutions within eXplainable AI (XAI) systems applied to time-series data in industrial contexts. Through this research, we have found that user-centered PD methodologies are important inclusions in designing feedback solutions for highly technical and complex industrial processes with XAI systems working with time-series data. By involving expert operators from the Kraft process in every step of the design process, we ensured that the feedback solutions were tailored to their specific needs, enhancing usability and relevance. Key recommendations include the need for immediate usable insights, model selection to enhance trust, quick and easy feedback interactions, and efficient interaction modalities. Our findings demonstrate the value of user-centered PD in minimizing unwanted features and aligning the design with industrial requirements. The insights gained offer a foundation for future research to adapt these recommendations to other industrial settings, contributing to the broader application of effective XAI interfaces and feedback solutions.