A GRPO-Based Approach to Hierarchical Knowledge Distillation in Military Information Extraction
Yi Yang, LI Zong-yong, Lingshu Li, Gaoshan Wang, Yu Hen Hu, Xia Peng, Yunxiao Li · IEEE Access · 2025
Military text processing faces multifaceted challenges, including real-time response requirements, high recognition accuracy demands, and computational resource constraints, compounded by semantic ambiguity, data scalability, and complex relational characteristics. To address these issues, this paper proposes a hierarchical knowledge distillation framework for military information extraction, leveraging the Group Relative Policy Optimization (GRPO) algorithm. The framework establishes a hierarchical knowledge transfer architecture, integrating a pre-trained large language model (Teacher Model, TM) with a lightweight model (Student Model, SM). Initially, systematic prompt engineering templates are designed to guide the TM in generating high-quality military text annotation corpora. Subsequently, semantic features from generative data are fused with a military domain knowledge base to construct an enhanced dataset. The GRPO algorithm is then employed to drive Low-Rank Adaptation (LoRA)-based fine-tuning of the SM on this enriched military corpus. Experimental results demonstrate significant improvements in military information extraction: the GRPO-optimized SM (SM-GRPO) achieves a 48.8% absolute increase in F1-score compared to the baseline SM, while reducing model parameters by 90.2% and inference latency by 83.7%. This approach effectively balances model compression and computational efficiency, offering a practical engineering solution for resource-constrained military environments.