LatEst: Vertical elasticity for millisecond serverless execution
Yannis Sfakianakis, Manolis Marazakis, Christos Kozanitis, Angelos Bilas · 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid) · 2022
Current state-of-the-art serverless frameworks can-not execute functions within a few milliseconds for bursty work-loads. The reason for that is that typically they rely on horizontal elasticity to cope with the varying demand for resources, which induces high overhead in the event of a cold -start. Recent literature has focused on minimizing the overhead of horizontal elasticity using mechanisms such as snapshots. However, the spawning of new function instances still requires several tens of milliseconds. This paper proposes vertical elasticity to scale resources of serverless functions to cope with bursting workloads. We design LatEst, a controller for serverless frameworks that adapts the allocated resources of active function instances. U sing vertical scaling, LatEst can adjust to bursts of function invocations within a few milliseconds. LatEst implements a feed-back control loop to: (1) predict the required resources during workload changes and (2) react rapidly and accurately to such changes. Moreover, LatEst spawns new instances for functions when the resources of the underlying server are reaching their limit. We evaluate LatEst as an extension of vHive [1] and find that LatEst can improve tail latency of serverless functions up to 25x compared to vHive.