OMNI at the Edge
Elizabeth Bautista, Nitin Sukhija, Melissa Romanus, Thomas Davis, Cary Whitney · 2022
With the increasing scale of High Performance Computing (HPC) systems and their support infrastructure coupled with the growing number of sensors and systems monitoring software, a deluge of operational data has become available for analysis and real time observation. NERSC uses a data warehouse called Operations Monitoring and Notification Infrastructure (OMNI) to store this data. However, there are challenges to gathering heterogeneous data from multiple sources such as latency, bandwidth, privacy, infrastructure or availability. Considerations for cybersecurity also need to be addressed. Some challenges associated with mitigating this data and making the data more available to users are resolved using edge computing technologies. This chapter will discuss how NERSC uses edge computing services to securely send data to OMNI from the heterogeneous sources available throughout the data center. The chapter will provide examples of how we use edge computing, our successes and challenges implementing the solutions and the prospective future work planned.