Programming parallel paradigms using selected APIs
Paweł Czarnul · 2018
This chapter proposes a code (available in file master-slave-MPI.c) of a basic master-slave application implemented with C and MPI. OpenMP allows implementation of master-slave type parallel programming. It adopts the very traditional implementation of the master-slave in which the master thread generates input at runtime and also merges results at runtime. A disadvantage of this approach is active waiting involved. Locks for input and output are used. This version is improved further on. The traditional master-slave scheme can also be implemented with MPI and Pthreads for hybrid environments, i.e., clusters with multiple cores within each node. MPI can be preferably used for communication between nodes, while Pthreads can be used to manage input data, computations and results across threads that run on cores within a node. The chapter also presents a basic master-slave scheme that can use many GPUs installed within a node. This version uses one host thread and the CUDA API for management of computations among GPUs.