Understanding the need for parallel computing
Paweł Czarnul · 2018
Parallelization of a sequential code may be easy as is the case, for example, in so-called embarrassingly parallel problems which allow partitioning of computations/data into independently processed parts and only require relatively quick result integration. For the past few years, increase in performance of computer systems has been possible through several technological and architectural advancements such as: within each computer/node: increasing memory sizes, cache sizes, bandwidths, decreasing latencies that all contribute to higher performance; among nodes: increasing bandwidths and decreasing latencies of interconnects; and Increasing computing power of computing devices. It can be seen that CPU clock frequencies have generally stabilized for the past few years and increasing computing power has been possible mainly through adding more and more computing cores to processors. This means that in order to make the most of available hardware, an application should efficiently use these cores with as little overhead or performance loss as possible.