Mind your Ps and Vs: A perspective on the challenges of big data management and privacy concerns
Divyakant Agrawal, Amr El Abbadi, Vaibhav Arora, Ceren Budak, Theodore Georgiou, Hatem A. Mahmoud, Faisal Nawab, Cetin Sahin, Shiyuan Wang · 2015
With large elastic and scalable infrastructures, the Cloud is the ideal storage repository for Big Data applications. Big Data is typically characterized by three V's: Volume, Variety and Velocity. Supporting these properties raises significant challenges in a cloud setting, including partitioning for scale out; replication across data centers for fault-tolerance; significant latency overheads due to consistency requirements; efficient traversal needs due to high update and velocity; and continuous maintenance in the presence of large variety of data representations. Last but not least, the storage of private data necessitates the ability to efficiently execute queries in a privacy preserving manner (P), without revealing user access patterns. In this paper we highlight these challenges and illustrate sample state of the art solutions.