Comparative Efficiency Evaluation of Hadoop and Spark Frameworks Using Random Forest Algorithm for Intrusion Detection
Wasnaa Kadhim Jawad, Abbas M. Al-Bakry · Ingénierie des systèmes d information · 2024
This study uses the Random Forest algorithm to evaluate the efficiency of Hadoop, and Spark distributed computing systems for intrusion detection, highlighting the growing importance of efficient distributed systems in handling big data.This research aims to assess and compare the performance of Hadoop and Spark in the context of an intelligent intrusion detection system.We use the Random Forest machine learning algorithm to train and test the system.The methods developed an intrusion detection system using Hadoop and Spark frameworks, followed by a thorough performance assessment using a real-world dataset.The problem this study tackles is the ever-increasing demand for processing data swiftly and accurately in a distributed fashion.We aim to identify the strengths and weaknesses of Hadoop and Spark in the context of machine learning-based intrusion detection.The "intelligent network detection system for intrusions" in this study uses a sophisticated security system using machine learning algorithms to detect potential intrusions, assessing Hadoop and Spark's performance in realworld scenarios and handling large-scale data processing.The findings provide insightful information about the efficacy and efficiency of distributed systems in machine learning activities, which can help select big data application frameworks.