Data mining method for anomaly detection in the supercomputer task flow
Vadim Voevodin, Владимир Валентинович Воеводин, Denis Shaikhislamov, Dmitry A. Nikitenko · AIP conference proceedings · 2016
The efficiency of most supercomputer applications is extremely low. At the same time, the user rarely even suspects that their applications may be wasting computing resources. Software tools need to be developed to help detect inefficient applications and report them to the users. We suggest an algorithm for detecting anomalies in the supercomputer’s task flow, based on a data mining methods. System monitoring is used to calculate integral characteristics for every job executed, and the data is used as input for our classification method based on the Random Forest algorithm. The proposed approach can currently classify the application as one of three classes – normal, suspicious and definitely anomalous. The proposed approach has been demonstrated on actual applications running on the “Lomonosov” supercomputer.