StarryStudioAI: Automating UI Design with Code-Based Generative AI and RAG
Bhavik Patel, Poojan Gagrani, Smeet Sheth, Kashish Thakur, Priya Varahan, Simon S.Y. Shim · 2025
Designing interactive and visually appealing web interfaces often requires extensive collaboration between designers and developers, making the process timeintensive and challenging. To address these limitations, the paper introduces StarryStudioAI, an innovative Large Language Model (LLM) powered plugin for Figma that translates natural language prompts into high-quality design outputs. Utilizing fine-tuned models, such as GPT-4o, Llama 3.2, and Gemini 1 Pro, enhanced with Retrieval-Augmented Generation (RAG), StarryStudioAI automates the generation of Hypertext Markup Language (HTML) and Cascading Style Sheets (CSS) code. It then seamlessly converts them into pixel-perfect layouts compatible with Figma. The system leverages a dataset of HTML/CSS samples derived from publicly available Figma-compatible templates and other open sources to ensure precision and reusability in generated code. Experimental results underscore StarryStudioAI's ability to achieve a CHaracterlevel F-score(ChrF) of 0.73, a BERTScore F1 of 0.79, and a Pass@1 rate of 0.33. To complement these metrics, a human evaluation involving 50 participants from technical and nontechnical backgrounds demonstrated high user satisfaction, with acceptability rate above 0.90 for component level designs (e.g., button) and satisfaction rate above 0.80 for layout consistency across UI components. These results validate the semantic and structural robustness of the generated designs while requiring minimal manual intervention. By streamlining design-to-development workflows, StarryStudioAI bridges the gap between textual specifications and visual outputs, enhancing productivity in web development through automated template generation and accelerated design-developer handoffs.