Partial AUC Maximization for Security Log Analysis Robust to Overfitting and Noisy Labels
Taishi Nishiyama, Atsutoshi Kumagai, Akinori Fujino, Kazunori Kamiya · 2024
To mitigate the damage caused by malware, network log analysis with machine learning for detecting suspicious logs has been attracting attention. In actual security operation, the true positive rate (TPR) in settings with a low false positive rate (FPR) is important since network operators must detect as many suspicious logs as possible while suppressing false positives. This paper focuses on a partial area under the curve (pAUC) maximization method that directly maximizes the TPR in an arbitrary FPR interval. If the FPR interval is set to a very small and narrow range when using the existing pAUC maximization methods in actual operation, the amount of data of benign logs to be trained is relatively limited. Therefore, if the FPR interval you aim to learn contains malicious logs that have been mislabeled as benign, the classification performance of the existing pAUC maximization methods can significantly deteriorate. In addition, when network logs are converted into feature vectors, the number of features tends to be large, which can lead to overfitting due to the relatively limited amount of data compared to the number of features. To alleviate these problems, we propose a method that combines the AUC maximization and pAUC maximization methods in accordance with the mathematical characteristics of the features. We also demonstrate the effectiveness of proposed method through comparative experiments with proxy logs from a real-world large enterprise network.