Empirical Analysis of the Fine-Tuning for Unsupervised Anomaly Detection in the ICT System
Yoichi Matsuo · 2023
Many unsupervised anomaly detection (UAD) methods for ICT systems that use only normal data collected from routers or servers, such as traffic volumes and text logs, have been developed to detect anomalies. Since the normal states of the ICT systems change due to the addition or deletion of devices, configuration changes, and OS updates, system operators need to collect enough normal data to learn various normal states whenever there is a change. Therefore, UAD methods cannot be applied until new data is gathered and a new normal state is identified. Using fine-tuning, one of the transfer learning techniques might reduce the duration of collecting normal state data. However, most of the existing papers focus on transferring knowledge of supervised models on image classification tasks, which are completely different from UAD in ICT systems. This paper analyzes a fine-tuning architecture to improve UAD in an ICT system with a small amount of normal data. By preparing five datasets, comprehensive experiments were conducted, and it was found that the fine-tuning architecture has the possibility to improve the accuracy of anomaly detection with a small amount of data for the ICT system dataset and scenarios.