LSTM-Based Memory Profiling for Predicting Data Attacks in Distributed Big Data Systems
Santosh Aditham, Nagarajan Ranganathan, Srinivas Katkoori · 2017
With the increase in sensitivity of data, big data clusters are a primary target for attackers. Most attack prevention and detection methods concentrate on network level analysis. In this paper, we propose a run-time profiling method to predict the memory usage of datanodes that can help in taking necessary actions in advance when there is a data attack in the big data cluster. Our proposed method relies on replication property of big data, models memory mappings as four-dimensional time series data and applies LSTM recurrent neural networks for prediction the memory usage. There are two steps in our proposed method: (1) Local Analysis where an LSTM (Long short-term memory) cell of a datanode predicts its memory usage, and (2) Consensus among replica datanodes regarding any abnormal behavior. We present the results of our proposed method by testing it on Hadoop MapReduce examples and show that our proposed method can successfully predict memory usage of a datanode with an error rate in the range 8-20 for inputs in the range 150-8000.