Transparent Load-Balancing for Network Intrusion Detection Systems
Matthias Vallentin · 2006
Since the amount of network traffic continuously increases, security analysis in largescale environments faces new challenges to keep pace with the rapidly growing traffic volume. Network intrusion detection systems (NIDS) form an integral part to secure the network perimeter by steadily inspecting network traffic in order to detect security breaches. The traditional single-machine architecture of a NIDS cannot provide enough resources to cope with the growing traffic volume. Vendors offer expensive solutions based on custom hardware. Aimed at high-performance environments, these systems seem to sustain the induced load, however, they fall short in providing a cost-effective and flexible solution. With our work, we set out to combine the performance of custom hardware with the flexibility and cost-efficiency of standard hardware. By distributing the network traffic stream over an expandable array of machines, we build a NIDS cluster suited for highperformance environments. In their entirety, the machines form a transparent NIDS cluster based on commodity hardware. To distribute the work load, each instance of the NIDS conducts analysis on a disjunct subset of the network traffic. A key challenge remains the exchange of lacking decision context. Facing this challenge, we discuss important design guidelines for NIDS clusters that promote the construction of effective implementations for practical use. Furthermore, we thoroughly evaluate our cluster with respect to accuracy and performance. Having a reliable testbed in place, we perform various measurements which yield insightful results. Based on our observations, we draw the conclusion that our approach provides a viable solution for large-scale networks. This thesis begins with a recapitulation of basic concepts of network intrusion detection. Particularly, we emphasize the main subject of our studies, the open-source NIDS Bro. After presenting its architecture and communication framework which our work inherently relies on, we highlight challenges that high-performance environments pose. Thereafter, we analyze objectives and mechanisms relevant for transparent loadbalancing. Finally, we present our transparent load-balancing NIDS cluster, operating in a large-scale research network at the Lawrence Berkeley National Laboratory.