Traffic Anomaly Detection at Fine Time Scales with
Jeff Kline, Sangnam Nam, Paul Barford, David Plonka, Amos Ron · 2008
Traffic anomaly detection using high performance measurement systems offers the possibility of improving the speed of detection and enabling detection of important, short- lived anomalies. In this paper we investigate the problem of detecting anomalies using traffic measurements with fine-grained timestamps. We develop a new detection algorithm (called S3) that utilizes a Bayes Net to efficiently consider multiple input signals and to explicitly define what is considered anomalous. The input signals considered by S3 are traffic volumes and correlations between ingress/egress packet and bit rates. These complementary signals enable identification of an expanded range of anomalies. Using a set of high precision traffic measurements collected at our campus border router over a 10 month period and an annotated anomaly log supplied by our network oper- ators, we show that S3 is highly accurate, identifying 86% of the anomalies listed in the log. Compared with well known time series-based and wavelet-based detectors, this represents over a 20% improvement in accuracy. Investigation of events identified by S3 that did not appear in the operator log indicate many are, in fact, true positives. Deployment of S3 in an operational environment supports this by showing zero false positives during initial tests. I. INTRODUCTION Whether malicious or unintentional, traffic anomalies are a fact of life in wide area networks. At a high level, the impact of anomalies is to reduce network performance and reliability, and as such, they are the bane of network operators. The standard process for addressing network anomalies is detect - diagnose - remedy. Therefore, improving any part of this process should improve network performance and reliability. The general objective of our work is to improve the ability to detect network traffic anomalies in operational networks. While a wide variety of both ad hoc and automated methods for detecting anomalies are currently used, two of the most important requirements for detection are accuracy and timeli- ness. An accurate detection method raises an alert if and only if an anomalous event occurs in the network. Likewise, a timely detection method raises an alert soon after an anomalous event begins. There are many challenges to accurate and timely anomaly detection. First, anomalies are difficult to define. While at- tacks, outages, flash crowds and misconfigurations are ex- amples of anomalies writ large, specifics vary from network to network depending on operational policy. Second, non- anomalous traffic has complex characteristics. The inherent