Enhancing Security Insights with KnowGen-RAG: Combining Knowledge Graphs, LLMs, and Multimodal Interpretability
Arnav Sharma, Khandakar Ashrafi Akbar, Bhavani M. Thuraisingham, Latifur R. Khan · 2025
We present a hybrid Retrieval-Augmented Generation (RAG) framework KnowGen-RAG that integrates knowledge graphs comprising entities and relationships and LLM-based Natural Language Generation for application security, privacy, and compliance. The framework aims to enhance the accuracy and relevance of retrieved information to produce more context-aware and actionable security recommendations, identify potential privacy risks, and detect vulnerabilities by utilizing structured knowledge about entities (e.g., access control mechanisms, security policies, privacy-preserving algorithms, protocols, software vulnerabilities) and their inter-relationships in the security context. We also extend the multimodal-LLM interpretability paradigm by contextual explanation generation for equations and tables from unstructured and highly technical documents. KnowGen-RAG proves superior for security-related information retrieval and contextual reasoning. It significantly outperforms both the LLM's output without RAG and Baseline RAG in terms of preciseness and reliability. Specifically, KnowGen-RAG increases accuracy for the CyberMetric dataset, where the original approach struggled to perform consistently for lightweight LLMs, and Baseline RAG achieved only marginal improvements. Additionally, KnowGen-RAG enhances answer quality for our curated security dataset, SecMD, demonstrating its effectiveness and improved understanding of security-related techniques and digital artifacts when addressing complex questions. The system aims to strengthen the learning of security professionals by providing thorough insights into the security landscape, encouraging informed decision-making in the face of sophisticated challenges.