Fine-grain protocol execution mechanisms and scheduling policies on smp clusters
Babak Falsafi, David Alvra Wood · 1998
Symmetric multiprocessor (SMP) clusters are emerging as the cost-effective medium- to large-scale parallel computers of choice, exploiting the superior cost-performance of SMP desktops and servers. These machines implement communication among SMP nodes by sending/receiving messages through an interconnection network. Many applications and systems use a variety of software protocols to coordinate this communication. As such, protocol performance can significantly impact communication time and overall system performance. This thesis proposes and evaluates techniques to improve fine-grain software protocol performance. Rather than provide embedded network interface processors, some systems schedule and execute the protocol code on the SMP processors to reduce hardware complexity and cost. This thesis evaluates when it is beneficial to dedicate one or more processors in every SMP to always execute the protocol code. Results from simulating a fine-grain software distributed shared memory (DSM) indicate that a dedicated protocol processor: (1) benefits light-weight protocols much more than heavy-weight protocols; (2) benefits systems with four or more processors per node; (3) will also result in the best cost-performance when scheduling overheads are much higher than protocol weight. Much like ordinary application software, the protocol code can execute either sequentially or in parallel. The central contribution of this thesis is a novel set of mechanisms, parallel dispatch queue (PDQ), for efficient parallel execution of fine-grain protocols. PDQ is based on the observation that by partitioning system resources among protocol threads, multiple threads can execute in parallel each accessing an exclusive set of resources thereby obviating the need for synchronization. This thesis proposes two fine-grain DSM systems--Hurricane and Hurricane-1--which execute software coherence protocols in parallel using PDQ. Hurricane achieves high performance by integrating embedded protocol processors into a network interface device. Hurricane-1 reduces cost by using SMP processors to execute the software protocol. Simulation results comparing the Hurricane systems to an all-hardware DSM implementation indicate that: (1) PDQ helps significantly improve software protocol performance; (2) Hurricane with four embedded processors performs as well as an all-hardware implementation; (3) Hurricane-1 performs within 75% of an all-hardware implementation on average.