System Abnormality Detection by Deep Learning and Joint Histogram Analysis
Chang‐Chieh Cheng, Chien‐Hsing Lin · 2024
Detecting system abnormalities is crucial for ensuring the security and stability of information systems. Traditional abnormality detection methods rely on analyzing system logs generated during system operation. However, recent advancements in deep learning have led to the development of novel log analysis techniques. Despite their efficacy, existing methods may misclassify logs with abnormally long lengths as anomalous. This paper in-troduces a novel approach that transforms an encoded log vector into a two-dimensional table using joint histogram analysis. By leveraging an autoencoder with multiple two-dimensional convolution layers, our method constructs a domain that characterizes the distribution of normal cases. Abnormalities are then identified through the application of a one-class classification method. Experimental evaluations conducted on three diverse log datasets collected from various information systems demonstrate the su-perior performance of our proposed method compared to three previous deep learning techniques.