A Retrieval-Augmented Generation Framework Based on a Knowledge Graph of Cybersecurity Vulnerabilities in Power Networks
Xingzheng Gao, Xing Chang · IEEE Access · 2025
To address the knowledge blind spots and hallucination issues of large language models (LLMs) in power grid cybersecurity, and to enhance their accuracy and practicality in decision support, this paper proposes a retrieval-augmented generation (RAG) framework based on a power grid security vulnerability knowledge graph. The framework integrates seven key steps—knowledge modeling, knowledge extraction, knowledge storage, vector indexing, intent recognition, similarity retrieval, and problem solving—to expand the contextual information of LLMs and enable enhanced generation. Experimental evaluation demonstrates the effectiveness of the proposed framework. Across six metrics—Answer Relevancy, Faithfulness, Context Precision, Context Recall, Answer Correctness, and Answer Similarity—the framework achieves scores of 92.41%, 91.06%, 98.17%, 97.28%, 94.78%, and 93.39%, respectively, outperforming existing RAG methods. In addition, performance tests show that, with relatively low generation time and token consumption, the framework achieves promising results in vulnerability verification and repair tasks, with verification and repair rates reaching 84% and 92%, respectively. Ablation studies further reveal that the problem-solving module has the greatest impact on answer relevance and fidelity, reducing them by 50.28 and 45.11 percentage points when removed. Similarly, the number of retrieved documents and similarity retrieval significantly affect contextual precision and recall, with the absence of similarity retrieval causing decreases of 18.17 and 22.28 percentage points. Qualitative assessments indicate that the framework performs well in mitigating hallucinations, handling global problem-solving, improving temporal adaptability, and enhancing interpretability, thereby demonstrating strong problem-solving capabilities. By integrating knowledge graphs with LLMs, this work not only offers a new approach for addressing practical issues but also contributes to the advancement of the power industry.