Design and Implementation of Online Log Anomaly Detection Model based on Text CM and Hierarchical Attention Mechanism
Yixin Zhou · 2025
With the increasing complexity of cloud computing systems and the explosive growth of log data scale, traditional anomaly detection methods that rely on manual feature engineering are no longer able to cope with high-dimensional temporal correlations in unstructured logs. Although deep learning technology can automatically capture complex patterns, it is limited by the scarcity of annotated data and the neglect of log event correlations. In response to this challenge, this article proposes a semi supervised log anomaly prediction model SSLTCNA, which achieves efficient and accurate detection by integrating a small amount of labeled and massive amounts of unlabeled log data. The model first constructs a log parsing and vectorization module to extract temporal semantic features. Secondly, a Transformer based log pattern learning network is designed to capture long-range dependencies. Finally, an improved MixMatch framework is introduced, which utilizes consistency regularization and pseudo label generation strategies to improve the utilization of unlabeled data. Experiments have shown that SSLTCNA performs excellently in unstructured logs and large-scale unlabeled scenarios: under supervised mode, the TCNA module has a detection accuracy of up to 99.98%, while the semi supervised version based on MixMatch only requires 10% labeled data to achieve a prediction accuracy of 99.57% (close to the performance of fully supervised models). At the same time, the training efficiency is improved by 40%, and the robustness to log format offset and noisy data is improved by 23% compared to the unsupervised baseline. Future research can further explore dynamic log parsing, real-time model updates, and semi supervised frameworks driven by reinforcement learning to promote the deep application of semi supervised technology in industrial grade log analysis.