Scampi - A Scaleable monitoring platform for the Internet
Jan Coppens · Ghent University Academic Bibliography (Ghent University) · 2004
In this paper we describe the architecture of SCAMPI (A Scaleable Monitoring Platform for the Internet). SCAMPI allows easy writing of monitoring applications, which can run on top of different network adapters without changing the code and which can provide detailed monitoring of high-speed Internet circuits. This is made possible by MAPI (Monitoring API) and the SCAMPI adapter, a programmable hardware monitoring adapter with built-in monitoring functionality. 1 Monitoring of high-speed networks Most backbone Internet circuits currently operate at speeds ranging from 1 Gb/s to 10 Gb/s. In order to verify operational, performance and security characteristics of the network and to enable problem resolution we need a high-speed network monitoring system. We need to measure elementary network performance characteristics, such as throughput, delay, packet loss rate and jitter. And we also need to search for traffic patterns indicating possible security problems, such as intrusion or denial of service attacks. Finally, we need a platform for creation of monitoring applications that can provide a view on network state at higher level of abstraction based on network monitoring. We can recognize three types of network monitoring according to how information about network is obtained: • Processing data from network components (e.g., SNMP counters and Netflow records) • Active monitoring, which injects testing packets into the network and processes them as they are received in another part of the network • Passive monitoring, which analyzes existing traffic in the network All types of monitoring have their advantages and difficulties. Processing data from network components provides continuous per-hop information, but tends to be unreliable due to problems with router software. Active monitoring is the easiest way to measure one-way delay, but it is generally unsuitable for other network characteristics, as it measures characteristics experienced by testing packets, rather than by existing traffic. Therefore, passive monitoring, which does not influence existing traffic, has become a popular method of precise and reliable network monitoring. However, passive network monitoring is becoming increasingly demanding on computing resources. The reason is that the physical network speed tends to increase faster than the computer processor speed. We already cannot monitor current high-speed network links just by tapping traffic with a regular network adapter, catching all packets with tcpdump and processing them even on the most powerful PCs. 2 SCAMPI architecture SCAMPI is a two-and-a-half year European project to develop a scaleable monitoring platform for the Internet. SCAMPI concentrates on passive monitoring. It has two main goals: • To enable easy writing of portable monitoring applications Application 1 Application 3 Application 2 Adapter 1 Adapter 2 Adapter 3 MAPI implementation Figure 1. SCAMPI architecture • To enable detailed monitoring of high-speed Internet The first goal is realized by providing MAPI Monitoring API, which enables application developers to start at higher level of abstraction of flows and monitoring functions. The second goal is realized by performing certain timecritical functions needed for most monitoring tasks inside the SCAMPI adapter, a specialized programmable monitoring adapter. The data rate going further to the host computer is thus significantly reduced. The project development effort therefore includes, going from bottom up, the SCAMPI adapter, MAPI and monitoring applications. SCAMPI architecture is illustrated in Fig. 1. Several applications run concurrently on top of MAPI, which in turn runs on top of various network adapters. Currently, we support the SCAMPI adapter, DAG adapters and regular Ethernet NIC cards. Owing to the modular MAPI implementation, support for other network adapters can be added easily. Therefore, applications are portable between computers equipped with any of these adapters. When certain adapter provides some monitoring function in its hardware or firmware, MAPI will automatically use it. If it is not provided by the adapter, MAPI will use its own software implementation of the particular function. We will describe individual components of the SCAMPI architecture in more detail, going again from bottom up.