Architecture to Speed up Redundant Data Processing in a MapReduce Cloud
Tzu‐Chi Huang, Kuo Chih Chu, Xue-Yan Zeng, Fu-Lin Jou, Ming‐Fong Tsai, Ce-Kuen Shieh · 2019
A cloud owns many resources usable to applications, but economically utilizing or saving the resources always is an important issue on cloud computing nowadays. A cloud probably wastes resources to do extra unnecessary computations due to processing redundant data such as MapReduce applications calculating the page ranking of hot news or rendering map zooms of a geographic location. Now, a cloud can utilize the Architecture to Speed up Redundant Data Processing (ASRDP) proposed in this paper in order to speed up the processing of redundant data in MapReduce. With the help of ASRDP, A cloud can greatly save the execution time of an application that processes redundant input data, especially when the application has high complexity in its Map function. According to the prototype implementation and related experiments with several popular applications in this paper, a cloud utilizes ASRDP to not only greatly improve performance when processing redundant input data but also intelligently avoid overheads of ASRDP when non-redundant input data.