CLogLLM: A Large Language Model Enabled Approach to Cybersecurity Log Anomaly Analysis
Hengyi Ren, Kun Lan, Zhi Sun, Shan Liao · 2024
Log data is commonly used to record the status of the system and the events that occur, which are often used to assist security staff to determine whether the system is abnormal, diagnose problems, troubleshooting. However, with the increasing complexity of the system, the log data becomes more and more numerous and complex, Therefore the analysis of the log becomes extremely difficult. Not coincidentally, with the emergence of various system software updates, traditional deep learning-based anomaly detection methods, which rely on pre-existing data for training, often struggle to maintain their effectiveness. At the same time, simply detecting anomalies in significant number of complex log data is no longer meet operational needs. Therefore, LLM which is capable of realizing zero-shot learning and text generation has attracted attention. This paper proposed method which is based on the implementation of LLMs for security log detection and analysis. The approach involves fine-tuning the LLMs to improve their adaptability to complex and diverse log data. Moreover, It incorporates two advanced prompt engineering techniques, ToT and Self-Refine, while also exploring the combination of existing techniques such as CoT and In-Context prompting. These three different prompt engineering methods significantly enhance the LLMs' the accuracy in log anomaly detection and the quality in generation of log anomaly analysis.