Monitoring Infrastructure: The Challenges of Moving Beyond Petascale
Amanda Bonnie, Mike Mason, Daniel Illescas · 2017
Scaling clusters is no longer the only struggle in moving towards exascale in HPC. While scaling components such as the network and file systems is a widely accepted need, monitoring, on the other hand, is often left behind in the procurement of these large systems. Monitoring is often quite an afterthought that is expected to be incorporated in existing infrastructure. While that often works for small systems, even petascale systems are starting to push the capabilities of monitoring infrastructure and their ability to collect and analyze complete system wide logs. The need and desire to do more cross-component relations will only get more complex with scale. Preparing for monitoring an exascale class machine is no small task.This paper presents the current redesign of our commodity monitoring infrastructure, the upgraded sub-system monitoring for Trinity, and ideas and concepts for moving towards exascale class monitoring.