A Comparative Study of Large-Scale Cluster Workload Traces via Multiview Analysis
Li Ruan, Xiangrong Xu, Limin Xiao, Feng Yuan, Yin Li, Dong Dai · 2019
Understanding the characteristics of workloads of the large-scale Clusters is the key to diagnose the system bottlenecks, making optimal configuration decisions, improving the system throughput and resource usage. Due to the diversity and multiview of the workload traces, featuring the good designs and bottlenecks by analyzing the workloads under realworld scenarios becomes increasingly challenging. This paper introduces a multiview based trace comparative analysis method by comparatively characterizing how the architecture, jobs, tasks, machines, and resources usage were managed among different platforms. A case study is performed which verified the effectiveness of our method by comparatively analyzing two most representative big traces: Google trace and Alibaba 2018 trace. Quantitative findings, together with the performance bottleneck inferences and suggestions are also presented. To the best of our knowledge, we are the first to perform such comparative empirical study on these two traces using a multiview based approach. Our multifaceted analyses and new findings not only reveal insights that we believe are useful for system designers, IT practitioners and users, but also can promote more researches on big trace data comparative analysis in large-scale clusters.