Analyzing structural similarity of user interface layouts of Android apps using deep learning
Nathalia da Cruz Alves, Leonardo Kreuch, Christiane Gresse von Wangenheim · 2022
The layout of graphical user interfaces (GUIs) takes into consideration the distribution of buttons, images, texts, and other components that are important factors for a positive experience of interactivity and usability. Therefore, it may be helpful to consult similar layouts during the design of GUIs for inspiration as well as to direct the development of original interfaces. This may be even more important in the context of the design of mobile applications in an educational context or when being developed by end-users with low-code environments, such as App Inventor. Therefore, this article presents an approach for analyzing the similarity of GUI layout using deep learning. First, GUI components and their positions are detected using the YOLO model. Then, the similarity of layouts is analyzed based on the GUI components detected. Results of the evaluation of the developed approach indicated a good accuracy of the detection of GUI components and the analysis of similarity close to human predictions. The results of this research can be used for the automated analysis of the similarity of GUI layouts of mobile applications, which can be used to analyze the design of mobile applications (consistency), search for similar designs, or assess the originality of a GUI design. It can also be applied in an educational context to assess the development of creativity as part of the development of mobile applications in order to teach computing.