Feedback-to-Text Alignment: LLM Learning Consistent Natural Language Generation from User Ratings and Loyalty Data
Zhenyu Gao · 2025
As large-scale pre-trained language models (LLMs) continue to deliver groundbreaking advances in natural language generation (NLG), a persistent challenge remains: how to reliably align model outputs with diverse and evolving human preferences in real-world, consumer-facing applications. While supervised fine-tuning (SFT) can improve base-model performance, it often fails to capture nuanced or shifting user tastes. Reinforcement Learning from Human Feedback (RLHF) introduces direct user signals but can be data-hungry and slow to adapt across multiple contexts. To address these limitations, we propose a novel Feedback-to-Text Alignment framework that leverages both immediate user ratings and long-term loyalty metrics to steer NLG toward more consistent, preference-aligned outputs. In summary, by unifying loyalty-driven reward signals, metareinforcement learning adaptability, and efficient soft-prompt compression, our Feedback-to-Text Alignment framework offers a scalable path toward producing commercially viable, user-aligned NLG in diverse real-world settings.