A Deep Learning Based Object Detection System for User Interface Code Generation
Batuhan Aşıroğlu, Sibel Senan, Pelin Görgel, M. Erdem Isenkul, Tolga Ensarı, Alper Sezen, Mustafa Dağtekin · 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA) · 2022
The Graphical User Interfaces (GUIs) of web applications include visuals and designs that allow users to interact with machines. Once the GUI design is done, it is necessary to generate its GUI code. However, the GUI code generation process is highly time consuming as well as highly dependent on software developers. Therefore, the development of automatic GUI code generating systems is of great importance recently. In this study, a GUI code generating system for web sites is designed using the Deep Learning (DL) approach. The dataset including “coordinate, width, height and type” of GUI objects is created using 7500 webpages. The created dataset is applied to the proposed system in order to detect objects in the GUI image and generate DSL mark-up code. Experiments were carried out to analyze the effectiveness of the proposed system and the performance evaluations were made.