Automated assessment of visual aesthetics of Android user interfaces with deep learning
Adriano Luiz de Souza Lima, Osvaldo P. Heiderscheidt Roberge Martins, Christiane Gresse von Wangenheim, Aldo von Wangenheim, Adriano Ferreti Borgatto, Jean Carlo Rossa Hauck · 2022
Visual aesthetics are seen as an essential success factor for mobile apps, affecting user experience and perception, which makes their evaluation crucial as part of the interface designing process. Machine learning approaches have shown to be very promising in this context, but so far, there are only solutions for assessing the aesthetics of web-based graphical user interfaces (GUIs). This article presents a deep learning approach to automatically quantify the visual aesthetics of GUIs of Android apps developed with App Inventor, based on a convolutional neural network (CNN) and adopting a regression-based supervised learning approach. The performance results demonstrate that the CNN can learn to evaluate the visual aesthetics of GUIs with a mean squared error of 0.023 for the validation set and 0.017 for the test set. The model ratings are also highly correlated with human ratings (Spearman rank correlation coefficient ρ = 0.86 for the validation set and ρ = 0.95 for the test set). And the results of a Bland & Altman analysis show that more than 95% of them agree with the human ratings. These promising results indicate that the model can be an effective and efficient means to automate the visual aesthetics assessment during the GUI design process for mobile apps.