User interface image hierarchical structure generation and optimization based on improved transformer
Fei Shen · Systems and Soft Computing · 2025
With the full advent of the digital age, user interface design is facing unprecedented challenges in terms of large-scale generation and personalized adaptation. However, traditional methods show obvious deficiencies when handling complex semantic structures and high-level visual expressions. To this end, the research proposes a novel method for generating and optimizing the hierarchical structure of user interface images by deeply integrating visual feature extraction and semantic relationship modeling techniques. This method introduces the prior memory self-attention mechanism for the first time to enhance the long-term dependency construction and semantic consistency maintenance in structural modeling. Moreover, it integrates local convolutional features and global semantic relations in the Transformer architecture, effectively compensating for the multi-scale differences and structural ambiguity problems in the transformation from interface images to structural trees. The experimental results show that, compared with the traditional convolutional network model, the proposed method has significantly improved the full matching rate index, reaching up to 96.3 % at the highest. Furthermore, on the high-complexity subset UI-Hard, the structural depth error of the research method is controlled within 0.44, and the semantic consistency score was as high as 4.0, demonstrating strong robustness and generalization ability. From this, it can be known that by combining the local feature extraction ability of convolutional neural networks, the global attention mechanism of Transformers, and the prior memory self-attention mechanism, the precise transformation from user interface images to hierarchical structure trees can be achieved. The research provides new technical paths in model structure design, semantic fusion strategies, and interface structure understanding, and has significant theoretical value and application potential.