Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation
Yuhao Wang, Ruiyang Ren, Yucheng Wang, Wayne Xin Zhao, Jing Liu, Hua Wu, Haifeng Wang · 2026
Long-form question answering (LFQA) requires open-ended long-form responses that synthesize coherent, factually grounded content from multi-source evidence.This makes reinforcement learning (RL) reward design critical.The reward must be verifiable for faithful grounding and stable optimization.However, many standard rewards assume a unique target with an exact-match notion of correctness, which fits short-form QA and math but breaks in LFQA.As a result, current RAG systems still lack verifiable reward mechanisms, yielding unstable feedback signals and suboptimal optimization outcomes.We propose RioRAG, a framework for reinforced verifiable informativeness optimization.First, it defines informativeness as a measurable and externally verifiable objective for RL.Second, RioRAG uses nugget-centric verification with cross-source checks to enable self-evolution of smaller LLMs and to provide denser, actiondiscriminative rewards that mitigate reward sparsity and stabilize optimization.This formulation avoids handcrafted supervision for the policy model and strong teacher-model distillation, relying instead on externally verifiable feedback.Experiments on LongFact and RAGChecker show that RioRAG achieves higher factual recall and faithfulness, establishing verifiable reward modeling as a foundation for trustworthy long-form RAG.Our codes are available at https://github.com/RUCAIBox/ RioRAG.