Automatic methods for hiding latency in high bandwidth networks (extended abstract)
Matthew Andrews, Tom Leighton, Panagiotis Metaxas, Lisa M. Zhang · 1996
In this paper we describe methods for mitigating the degradation in performance caused by high latencies in parallel and distributed networks.Our approach is similar in spirit to the "complementary slackness" technique for latency hiding but has the advantage that the slackness does not need to be provided by the programmer and that large slowdowns are not needed in order to hide the latency.For example, given any algorithm that runs in T steps on an n-node ring with unit link delays, we show how to run the algorithm in O(T) steps on any n-node bounded-degree connected network with average link delay 0(1 ).This is a significant improvement over prior approaches to latency hiding, which require slowdowns proportional to the maximum link delay (which can be quite large in comparison to the average delay).In the case when the network has average link delay L&, our simulation runs in O(&Z') steps using n/G processors, thereby preserving efficiency.We also show how to simulate an n x n array with unit link delays using slowdown O(df& log513 n) Q '2/3 log-sla n)-nodearray with average link on an (n dave delay d~ve.We anticipate that our results wilf be of interest in the context of parallel and distributed computing on networks of workstations (N OWS).NOWS typically