Scalable Data Placement of Data-intensive Services in Geo-distributed Clouds
Ankita Atrey, Gregory Van Seghbroeck, Bruno Volckaert, Filip De Turck · 2018
The advent of big data analytics and cloud computing technologies has resulted in wide-spread research in finding solutions to the data placement problem, which aims at properly placing the data items into distributed datacenters.Although traditional schemes of uniformly partitioning the data into distributed nodes is the defacto standard for many popular distributed data stores like HDFS or Cassandra, these methods may cause network congestion for data-intensive services, thereby affecting the system throughput.This is because as opposed to MapReduce style workloads, data-intensive services require access to multiple datasets within each transaction.In this paper, we propose a scalable method for performing data placement of data-intensive services into geographically distributed clouds.The proposed algorithm partitions a set of data-items into geodistributed clouds using spectral clustering on hypergraphs.Additionally, our spectral clustering algorithm leverages randomized techniques for obtaining low-rank approximations of the hypergraph matrix, thereby facilitating superior scalability for computation of the spectra of the hypergraph laplacian.Experiments on a real-world trace-based online social network dataset show that the proposed algorithm is effective, efficient, and scalable.Empirically, it is comparable or even better (in certain scenarios) in efficacy on the evaluated metrics, while being up to 10 times faster in running time when compared to state-of-the-art techniques.