Automated usability evaluation utilizing log files and data mining techniques.
S M Shafaei · 2023
Usability evaluation is one of the essential aspects of software production. This evaluation should be done during the entire life cycle of a software application, from pre-production to production and post-production. However, the collection and evaluation of usability data can be a very challenging, time-consuming, and expensive task to be conducted manually, particularly for certain types of products and working conditions. These challenges may include the need to recruit participants fully engage and motivate them during evaluation, and factor in environmental conditions. Other challenges may include collecting data in real-world environments, especially when the users are geographically dispersed, minimizing evaluator and participant bias, and analyzing complex data sets, particularly when the volume of data is large. This research explores an alternative approach to automate the whole process of usability evaluation by utilizing data mining and machine learning techniques on data recorded in log files. The objective of this dissertation is to extract the value of usability-related metrics from user interactions and estimate usability measures in a quantitative manner. The use of log files for usability evaluation offers several advantages over traditional methods of evaluation, such as collecting data objectively without the need for subjective interpretation of the evaluator in some cases, creating a comprehensive view of user interactions with the possibility of identifying user behavioral patterns and trends that appear in large data sets, and collecting data from real-world scenarios instead of data from simulated scenarios. Other advantages include reducing evaluation costs, enabling remote data collection from anywhere in the world, which leads to the identification of location-dependent usability problems, and facilitating continuous monitoring over time, which leads to the identification of time-dependent usability issues. In this dissertation, Chapter II provides a comprehensive categorization, comparison, and summary of the pertinent usability evaluation techniques that utilize log files as input data in both academic and industrial research. Each method is examined carefully, and its respective strengths and weaknesses are highlighted to provide a systematic understanding of the advantages and limitations of each technique. Chapter III assesses the originality of the research questions and proposed solutions. Given the ongoing development and refinement of log file analysis tools and techniques by proficient teams in advanced corporations, and the substantial research already conducted in web usage mining, this dissertation compares its solution with prior works in these two domains. The similarities and disparities between them are evaluated to determine the uniqueness and value of