Distributed parallel computing with Futhark: a functional language to generate distributed parallel code
Michaël El Kharroubi, Baptiste Coudray, Orestis Malaspinas · 2022
In this paper, we present two proofs-of-concept for distributed-memory parallel approaches based on the Futhark functional programming language. Futhark is an array-based language generating high-performance code for CPU and GPU back-ends, leveraging shared-memory parallelization techniques. While the code generated by Futhark is extremely efficient, it lacks the capability to be distributed among several computing nodes, which is necessary in many engineering applications (computational fluid mechanics, meteorology, etc.). To this aim, it is desirable to add an MPI back-end to the Futhark compiler. In order to test the feasibility of a new compiler back-end, we implemented a C library wrapping Futhark kernels and handling a multi-block decomposition and communications. This library showed very promising performance and speedup results in the case of stencil-based algorithms. It thus allowed the initiation of the second part of our project: the implementation of a complete compiler back-end for the Futhark language. In this first attempt, we are using a naive memory model that has the advantage of simplicity at the cost of low efficiency. We show that we implemented most of the second-order array combinators of the language, which are the abstractions responsible for the vast majority of its parallelization capabilities, and we propose ways to go beyond our naive memory model.