Performance and scaling of locally-structured grid methods for partial differential equations - eScholarship
Phillip Colella, John B. Bell, Noel D. Keen, Terry J. Ligocki, Michael J. Lijewski, Brian Van Straalen · 2008
Performance and Scaling of Locally-Structured Grid Methods for Partial Differential Equations Phillip Colella, John Bell, Noel Keen, Terry Ligocki, Michael Lijewski, Brian Van Straalen Computational Research Division, Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720 E-mail: [email protected] Abstract. In this paper, we discuss some of the issues in obtaining high performance for block-structured adaptive mesh refinement software for partial differential equations. We show examples in which AMR scales to thousands of processors. We also discuss a number of metrics for performance and scalability that can provide a basis for understanding the advantages and disadvantages of this approach. 1. Introduction A broad range of applied PDE problems exhibit multiscale behavior, i.e. variation in the solution over scales that are much smaller than the global large scales in the problem. Examples include flame fronts arising in the burning of hydrocarbon fuels and nuclear burning in supernovae; in geophysical problems, ocean currents, effects of localized features in orography or bathymetry, and tropical cyclones; and in plasma physics, a variety of small scale effects due to nonlinear instabilities and localized kinetic effects. In all of these problems, the fundamental mathematical description is given in terms of various combinations of PDE of classical type (elliptic, parabolic, hyperbolic). To effectively compute solutions to such problems, we need simulation capabilities with the following features. • Multiresolution / adaptive methods: discretization methods that locally adjust the resolved length scales as a function of space, time, and the solution. • Semi-implicit or fully-implicit methods for computing long-time dynamics in the presence of stiff fast dynamics. • High-performance, scalable implementations. In grid-based methods for numerical PDE, it is necessary to make a fundamental choice of discretization technologies. We have chosen to use locally structured grids, i.e. ones that are based on defining discrete unknowns on a rectangular discretization of the spatial independent variables. Specifically, we mostly use the finite-volume approach, in which the rectangular grid defines a collection of control volumes, for which a natural discretization of the divergence operator is obtained by integrating over the control volume. This leads to methods that satisfy discrete conservation laws, an essential feature if one is computing discontinuous solutions to PDE, and a desirable property for a much larger class of physical problems. There is a large