Conducting usability evaluation of a multi-purpose device for expert users: Case ABB ship control interface
Niko Aho · Aaltodoc (Aalto University) · 2019
This master’s thesis studies usability testing in context of conducting tests with multi-purpose devices for expert users. More precisely, it focuses on usability testing with expert users in industrial usability research. Objective of this study is to research how to design usability testing process for expert users, how does the multi-purpose nature of the tested product affect testing, and how does the level of user expertise inflict objective and subjective usability in testing. The research consists of literature review on usability testing and a case study with a client company. The case study inspects usability of a multi-purpose marine controller interface, produced by ABB Marine & Ports Finland which is also the client company for the case study. The users participated in the study were navigation officers with varying number of years served in the domain. However, all of them can be classified as an expert user of controlling interface. In the literature review, this thesis inspects the current state of research on usability testing regarding subjects of this research and charts the positioning of this research based on theoretical frameworks. Then, the case study is analysed reflecting the results of the literature review on the process of conducting usability testing for ABB ship controller interface. This thesis suggests that expert users are capable of revealing context-specific usability issues based on their professional expertise although the testing would be conducted in controlled laboratory settings. The participants of the case study are likely to utilise the interface for multiple years without possessing alternatives to them which increases importance of device ergonomics in comparison to products utilised mainly by novice users. Additionally, they might not value learnability as main usability attribute and benefit more from effective and accurate devices which allow for utilising their domain expertise. The effect of multi-purpose interface in usability testing is reviewed based on the case study. This thesis suggests that multi-purpose prototypes require testing each function, although achieving complete fidelity of each function might not be possible. According to research, it might assist in recognising possible conceptual flaws and identifying improvements for them. Moreover, if multi-use concept consist of numerous purposes testing might require higher investment of resources in order to achieve satisfactory level of detail in test results. This denotes that scoping usability testing clearly is more important in case of multi-purpose devices than devices with fewer applications. The experiences from the case study suggest that this results in correct balance for number of tasks performed, task duration and number of methods for data gathering. The effect of level of user experience in usability testing is studied by comparing different sets of data provided by user groups separated by their years of practice in maritime domain. Users with more experience were more likely to provide direct feedback on usability issues they experienced. In addition, they were more likely to prefer devices they are more familiar with. This implies that users with more expertise are likely to adhere on their previous learning when testing new concepts, which will affect results of usability testing with them. User groups with less experience were more likely to prefer the new concept and expressed their opinions more carefully during the tests.