A strong law for the rate of growth of long latency periods in a cloud computing service

Abstract

Cloud-computing shares a common pool of resources across customers at a scale that is orders of magnitude larger than traditional multiuser systems. Constituent physical compute servers are allocated multiple `virtual machines' (VMs) to serve simultaneously. Each VM user should ideally be unaffected by others' demand. Naturally, this environment produces new challenges for the service providers in meeting customer expectations while extracting an efficient utilization from server resources. We study a new cloud service metric that measures prolonged latency or delay suffered by customers. We model the workload process of a cloud server and analyze the process as the customer population grows. The capacity required to ensure that the average workload does not exceed a threshold over long segments is characterized. This can be used by cloud operators to provide service guarantees on avoiding long durations of latency. As part of the analysis, we provide a uniform large deviation principle for collections of random variables that is of independent interest.