A Locality-aware Task Scheduling Algorithm for Video Transcoding over Heterogenous MapReduce Cluster
Xiangjun Deng, Jing Yu Huang, Renfa Li · 2018
MapReduce is emerging as a very promising technology for high computational task, for instance video transcoding. Because of the complexity of video decoding and encoding in transcoding, we divide the video transcoding job into several segments, then mapping these segments over heterogenous MapReduce cluster to handle. In this paper, we propose a locality-aware heuristic transcoding task scheduling algorithm to balance the segment transmission time and the task transcoding time, LA-MCT, which classifies all segments as local and remote segments, and then maps these segments to an optional machine using the MCT load balancing strategy. The optimal scheduling schema could be selected by traversing all local and remote segments pairs. Large number of numerical simulation experiments validate that the proposed algorithm can generate the minimum entire finish time compared with the existing algorithms.