An Introduction to Cloud Benchmarking
David Bermbach · 2015
Over the last few years, more and more Cloud Computing offerings have emerged ranging from compute, data storage, and middleware services over platform environments up to ready-to-use applications. Choosing the best offering for a particular use case, is a complex task which involves comparison and trade-off analysis of functional and non-functional service properties; for non-functional quality of service (QoS) properties, this is typically done via benchmarking. Today, a plethora of benchmarking solutions exist for different layers in the cloud stack (IaaS, PaaS, SaaS) which typically address a single QoS dimension - a holistic cloud benchmark even for a single layer in the cloud stack is still missing. In this tutorial, we will give an overview of existing cloud benchmarking solutions and point-out ways in which these different benchmarks could be used in concert to actually compare clouds as a whole (i.e., for instance Amazon cloud vs. Google cloud) instead of analyzing isolated QoS dimensions of single cloud services.