Revisiting Benchmarking Methodology for Interconnect Devices
Daniel Raumer, Sebastian Gallenmüller, Florian Wohlfart, Paul Emmerich, Patrick Werneck, Georg Carle · 2016
Ever growing demand for network bandwidth makes computer networks an area of constant development and fast adjustments. The steady change makes good performance assessments equally necessary and challenging. This development motivated us to revisit the established benchmarking methodology. We provide an overview of the state-of-the-art in router benchmarking, the currently available benchmarking tools, and challenges for benchmarks. A discussion of benchmarking results for three different devices (routers based on Linux and FreeBSD, and a MikroTik router) reveal different properties currently not covered by standardized benchmarks. We conclude by adding tests to the common benchmarking methodology reflecting these properties to make the results more valuable. The prototype software implementation of our own benchmarking tool and its measurement reports are publicly available.