Look at my Network: An Insight into the ISP Backbone Traffic
Tomáš Beneš, Jaroslav Pešek, Tomáš Čejka · 2023
High-speed ISP networks provide several challenges that prevent the creation of long-term datasets for giving insight into the traffic. Currently, there are no publicly available long-term datasets capturing the entirety of high-speed ISP networks. Such networks are traditionally monitored using IP Flows, which provide enough high-level information about the situation in the network and support various use cases, such as the detection of outages or security threats. Even with this type of aggregation long-term datasets are very unpractical due to their size. The other problem is that flow monitoring comes with significant aggregation and common traffic statistics are brief and lack useful details and require further processing. This paper addresses these problems and presents a new long-term aggregated dataset, a detailed analysis of public network traffic measured on the ISP backbone, and a monitoring architecture composed of open-source tools capable of using an existing flow exporter infrastructure. Such insight into traffic helps to design and develop hardware optimizations, tuning the performance of monitoring systems, and adapting security detection algorithms.