Acceptance Testing the Chicoma HPE-Cray EX Supercomputer
Jennifer Kathleen Green · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2024
Since the installation of MANIAC I in 1952, Los Alamos National Laboratory (LANL) has been at the forefront of addressing global crises using state-of-the-art computational resources to accelerate scientific innovation and discovery.This generation faces a new crisis in the global COVID-19 pandemic that continues to damage economies, health, and wellbeing; LANL is supplying high-performance computing (HPC) resources to contribute to the recovery from the impacts of this virus.Each system that LANL's HPC Division installs requires an understanding of the intended workloads for the system, the specifications and expectations of performance and reliability for supporting the science, and a testing plan to ensure that the final installation has met those requirements.Chicoma, named for a mountain peak close to Los Alamos, NM, USA, is a new HPC system at LANL purchased for supporting the Department of Energy's Office of Science Advanced Scientific Computing Research (ASCR) program.It is intended to serve as a platform to supply molecular dynamics simulation computing cycles for epidemiological modeling, bioinformatics, and chromosome/RNA simulations as part of the 2020 Coronavirus Aid, Relief, and Economic Security (CARES) Act.Chicoma is among the earliest installations of the HPE Cray EX supercomputer running the HPE-Cray Programming Environment (PE) for Shasta Architecture.Like other cutting-edge HPC systems, Chicoma is being installed in phases-beginning with the testbed (Chicoma) installation, followed by the delivery, installation, and integration of the final system.A fundamental part of the preparation of the system focuses on verifying that it can support the science for which it was purchased.To ensure a supercomputer's viability for real customer workloads, LANL HPC support staff plans, prepares, and conducts extensive testing.The Programming and Runtime Environments Team (PRETeam) of LANL's High Performance Computing Environments Group, supports HPC environments in production and facilitates the integration and acceptance testing of new systems.The Chicoma HPE-Cray EX system is evaluated by the PRETeam over its phases of installation to ensure that the Y Dakota State University Advanced Research Laboratory resulting product meets the requirements as set forth by the statement of work for the procurement.By generating substantive test results, we empirically demonstrate its ability to perform the intended scientific workloads.The testing plan developed for the system includes functional, performance, and application tests selected to quantify the ability of the system to meet performance, reliability, and precision requirements specifically for supporting the research efforts of the COVID-19 HPC Computing Consortium.The Pavilion HPC Testing Harness automates the execution of tests on HPC systems.Pavilion was totally redesigned from its first iteration by LANL's HPC PRETeam.These improvements are leveraged in the Chicoma test configurations to enable a simplified and reproducible benchmarking and synthetic workload.Developing and harnessing the tests under Pavilion ensures controlled consistency of the application's build and execution environments.Pavilion's test definitions express rules to generate scripts to interface the environment modulefiles, filesystem, scheduler, and system resources to build, schedule, run, collect, and log results for each test.The Splunk data analytics platform and programmed dashboards automatically graph results as they're fed into the system.This simplifies the process of analyzing and deriving statistics among tests of similar configurations, reducing time spent post-processing results.This paper documents the acceptance testing process executed by LANL HPC Environments Group's PRETeam staff to ensure that the Chicoma system meets the requirements of the funding program and is capable of sustaining a predetermined workload of synthetic and real scientific applications.It describes the Chicoma acceptance test suite, configurations and tuning of the tests under Pavilion, presents the results of acceptance testing, and concludes with a discussion of the outcomes of the acceptance testing effort on Chicoma and future work.