Cost-efficiency and Performance Robustness in Serverless Data Exchange
David Justen · Proceedings of the 2022 International Conference on Management of Data · 2022
Enterprises increasingly store their sporadically used data on cheap, elastic cloud storage to save costs. Analytical workloads on that data often appear in infrequent bursts presenting a high disparity in input data sizes. Using conservatively over-provisioned compute resources for this class of workloads is not cost-efficient as a fixed amount of resources only matches steady demand. Elastic query processors with a serverless architecture resolve this issue by running workers in cloud functions [3][9][10]. These systems can start thousands of functions within seconds, enabling elasticity down to query pipeline granularity. Moreover, serverless query processors come at no cost for idle times.