SAHWS:IoT-enabled Workflow Scheduler for Next-Generation Hadoop Cluster
Jahwan Koo, Isma Farah Siddiqui, Bhawani Shankar Chowdhry, Nawab Muhammad Faseeh Qureshi · 2022
Big data analytics processes large-scale datasets in the next-generation based distributed computing environment. This task is carried out through an ecosystem named Hadoop that schedules jobs of the cluster. The workflow scheduler of Hadoop manages homogeneous jobs in nature, which contradicts the modern storage-heterogeneity-aware jobs such as in-place jobs processed through RAM and SSD drive. This paper presents SAHWS (Storage-Aware Heterogeneous Workflow Scheduler), which facilitates job heterogeneity and coordinates with tasks, i.e., Map Reduce in a storage-aware environment. The proposed workflow scheduler fills the gap of communicating storage-aware heterogeneous jobs in the Hadoop cluster.