AI Enhanced Anomaly Detection of System Logs in Cyber Security
S. Ganesh Balaji, Dash Puspita, S Sriram, S Ragul · 2024
The detection of anomalies in log data is essential for safeguarding digital infrastructures, as it helps identify irregularities that could signal potential security threats. As the complexity and volume of log data grow, Security Operations Center (SOC) analysts encounter mounting difficulties in quickly and effectively identifying and responding to incidents. This paper reviews advanced log anomaly detection techniques that employ AI technologies, with a specific focus on the Isolation Forest Algorithm. We delve into the integration of Endpoint Detection and Response (EDR) tools, pivoting techniques, process tree analysis, and summarization methods to enhance the identification and interpretation of suspicious activities. The paper discusses the development and examination of process trees, aiming to equip SOC analysts with practical insights and recommendations. Additionally, we assess how AI-powered log analysis can overcome existing challenges and improve the detection of complex threats. Finally, we summarize our key findings and propose future research avenues to address ongoing challenges in log anomaly detection.