Randomized assignment of jobs to servers in heterogeneous clusters of shared servers for low delay

Abstract

We consider the problem of assignning jobs to servers in a multi-server system consisting of $N$ parallel processor sharing servers, categorized into $M$ ($\ll N$) different types according to their processing capacities or speeds. Jobs of random sizes arrive at the system according to a Poisson process with rate $N\lambda$. Upon each arrival, some servers of each type are sampled uniformly at random. The job is then assigned to one of the sampled servers based on their states. We propose two schemes, which differ in the metric for choosing the destination server for each arriving job. Our aim is to reduce the mean sojourn time of the jobs in the system.

It is shown that the proposed schemes achieve the maximal stability region, without requiring the knowledge of the system parameters. The performance of the system operating under the proposed schemes is analyzed in the limit as $N\to\infty$. This gives rise to a mean field limit. The mean field is shown to have a unique, globally asymptotically stable equilibrium point which approximates the stationary distribution of load at each server. Asymptotic independence among the servers is established using a notion of intra-type exchangeability which generalizes the usual notion of exchangeability. It is further shown that the tail distribution of server occupancies decays doubly exponentially for each server type. Numerical evidence shows that at high load the proposed schemes perform at least as well as other schemes that require more knowledge of the system parameters.