Procedural Game Level Generation Using Large Language Models and Reinforcement Learning with Human Feedback
Junbeom Lee, Jung In Kim, Jongkook Heo, Jungmin Lee, Jaehoon Kim, Seoung Bum Kim · Journal of Korean Institute of Industrial Engineers · 2025
Designing game levels during development is both time-consuming and resource-intensive. To address this challenge, recent studies have focused on automated level generation using deep neural networks trained through supervised learning. However, game level datasets suitable for training are often unavailable or contain structural errors that violate game mechanics, resulting in unplayable levels. To overcome these limitations, this study proposes Mario level generation using human preferences (MarioPref) that uses low-quality Super Mario Bros game level data and conditional prompts to generate playable levels that satisfy specified constraints. The framework uses a two-stage training approach: first, a large language model (LLM) is fine-tuned using supervised learning to reconstruct masked level elements; second, reinforcement learning with human feedback (RLHF) is applied to enhance playability and prompt adherence. Experimental results indicate that MarioPref effectively generates playable levels that satisfy input prompts, even when trained on low-quality data. These findings demonstrate the practical value of using human feedback to enhance content quality under data scarcity and suggests the potential for reliable content generation across diverse game genres.