SimLog: System Log Anomaly Detection Method Based on Simhash
Weiping Wang, Huijuan Wang, Yulu Hong, Chenyu Wang, Hong Tao Song, Shigeng Zhang · 2024
Enterprises face increasingly complex and frequent security threats, presenting significant challenges for timely prevention and response. Traditional log-based intrusion detection systems often rely on known attack signatures, limiting their ability to detect novel or evolving threats. Supervised anomaly detection methods, while leveraging machine learning techniques, are constrained by the scarcity of labeled attack samples, leading to gaps in detecting real-world attack variations. To address these limitations, this paper proposes a lightweight anomaly detection framework tailored for relatively stable server environments. The approach constructs provenance graphs from audit logs, extracts local subgraphs centered on process nodes, and utilizes Simhash for semantic embedding and frequency analysis. By combining locality-sensitive hashing with the K-medoids clustering algorithm, the method establishes a robust normal behavior model to detect anomalies. Experimental evaluations on public datasets and high-performance computing platforms demonstrate that the proposed method achieves 97% detection accuracy while significantly reducing the time costs compared to existing methods.