Hard Disk Failure Prediction on Highly Imbalanced Data using LSTM Network
Cahyadi Cahyadi, Matthew J. Forshaw · 2021 IEEE International Conference on Big Data (Big Data) · 2021
Failure prediction of hard disks has garnered significant interest from the research community in recent years. Several prior studies have leveraged operational datasets from large cloud storage providers to develop failure models which have been shown to exhibit good predictive accuracy. However, these datasets are highly imbalanced, with relatively scarce data concerning failed drives. In this paper, we set out to develop accurate predictions leveraging only commonly used S.M.A.R.T. attributes as predictors and to explore the impact of various imbalance mitigation strategies on our predictive ability. We leverage open data from Backblaze in developing an LSTM-based model which achieves a Matthews Correlation Coefficient of 0.71. We demonstrate the potential of more universally applicable models, portable to new operational datasets and disk types.