Network capacity for data intensive applications on clusters of workstations
Guy Pardon, Gustavo Alonso · Repository for Publications and Research Data (ETH Zurich) · 1998
Component software, distribution, and the use of clusters of workstations are all key trends in today's technology.Little attention has been paid, however, to the network bandwidth required for data intensive applications.In the context of databases, much w ork has been done in parallelization strategies for monolithic architectures with dedicated, specialized networks or over disk arrays.We e n vision, however, data intensive applications over clusters of PCs or workstations connected by a commodity LAN.For such scenarios, this paper evaluates and analyzes the network requirements of a data processing application built from stand-alone components connected by a LAN.Using Amdahl's law in the context of a serial run of complex data-manipulating tasks (TPC-D like), we derive an estimate of the required network bandwidth as a function of the number of nodes involved, disk I/O bandwidth, amount of main memory vs. data volume size, and the nature of the applications.The analytical results are validated with experiments running the TPC-D benchmark over a prototype implementation of such high-level parallel data management system.The results prove the feasibility of the data intensive cluster approach a n d m a y serve as a guideline for other systems based on clusters of workstations and stand-alone components.