Data Mining-Based Analysis of HPC Center Operations
Jannis Klinkenberg, Christian Terboven, Stefan Lankes, Matthias Müller · 2017
Size and complexity of contemporary High Performance Computing (HPC) systems increases permanently. While the reliability of a single component and compute node is high, the huge amount of components comprising these systems results in the fact that defects happen regularly. This drives the need to manage failure situations. Common issues are component failures or node soft lock-ups that typically lead to crashes of the user jobs that are scheduled on the affected node, and may cause undesired downtime. One approach to mitigate the impact of such problems is to predict node failures with a sufficient lead time in order to take proactive measures. However, accurate prediction is a challenging task.The literature describes several approaches that focus on gathering and analyzing system event logs in order to create prediction models. In this paper, we present a different approach by using descriptive statistics and supervised machine learning to create a prediction model from monitoring data. Our approach is based on the assumption, that features of a certain time frame before a critical event (i. e., a failure or soft lock-up) can serve as an indicator. Consequently, our model is trained with monitoring data from critical and healthy time frames. The evaluation with standard monitoring data collected from the HPC systems at RWTH Aachen University shows that our classifier is able to locate potentially failing nodes with a 10-fold cross precision of 98% and recall of 91 %.