Cloud SecNavigator: RAG Approach to Bridge Gaps and Strengthen Cloud Security Practices with RAGAS Assessment
Rei Watanabe, Satoshi Okada, Koki Watarai, Takuho Mitsunaga · 2024
In recent years, many cyber incidents have resulted from misconfigurations of AWS. Although AWS provides exten-sive security guidelines, the sheer volume of documentation makes it difficult for developers to read and apply them completely. To address this, we propose a development support tool, Cloud SecNavigator. This tool uses Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) to extract relevant content and accurately respond to user queries based on AWS documentation. Our evaluation measures Cloud SecNavigator's output accuracy using two approaches: (1) Retrieval-Augmented Generation Assessment (RAGAS) and (2) a comparative accuracy assessment between Cloud SecNavigator-generated responses and a non-RAG LLM (GPT-40). The results indicate that Cloud Sec-Navigator achieves superior accuracy, highlighting its potential as an effective development support tool.