A New Load Balancing Approach for Coupled Multi-Physics Simulations
Amin Totounferoush, Neda Ebrahimi Pour, Juri Schröder, Sabine P. Roller, Miriam Mehl · 2019
The simulation of multi-physics and multi-scale problems requires very highly scalable approaches to be efficient on today's supercomputers. Thus, the need of scalable approaches to realise large problems in a feasible time has to be addressed respectively. In this paper, we present an approach to run such complex simulations more efficiently by proposing a new load balancing approach across several solvers used in a partitioned simulation. We demonstrate the applicability of the approach for a small academic test case representing a fluid-acoustic coupling. The partitioned approach allows for a different treatment of each subdomain, hence the best-suited configuration for each of them. We show how well our approach is scalable by running scalability measurements on the SuperMUC supercomputer at the LRZ supercomputing center. The central issue in partitioned simulation that we address in this paper is the inter-solver load imbalance. We present a new method to efficiently distribute the total number of requested cores between the solvers to reduce or ideally remove the time that one solver needs to wait until the other one finishes its computation. To demonstrate the effectiveness of our proposed method, we use a simple Gaussian pulse located inside a cubic domain. The domain is decomposed into an inner and an outer subdomains, while the pulse is located in the inner part and it spreads over the time to the outer subdomain. In the inner domain, we solve the Euler equations while in the outer domain a linearized set of Euler equations is solved. We use a two-way explicit coupling, i.e., the inner part provides its values at the coupling interface to the outer subdomain and receives information from the outer subdomain. This procedure is done for the outer domain as well. Hence both domains provide and receive information from each other. It is obvious that, if one of the solvers is slower, the other one will be idle at the end of each time step. For communication and data exchange between the subdomains, we use the preCICE coupling library. Numerical results show that the presented framework can scale up to (at least) 560 cores. In addition, the proposed load balancing method is able to almost remove the load imbalance between the coupling partners. The effect of using this method is significant and in most cases that we simulated, the run-time could be reduced by more than 40 percents in comparison to the results of the old load-balancing scheme.